What does "think step by step" actually do to a reasoning model?
Adding "think step by step" to a reasoning model prompt buys you almost nothing in accuracy and costs you 20 to 80 percent more time. Reasoning models already generate an internal reasoning trace before answering. Your instruction duplicates work the model has already scheduled, and on some models it makes answers less consistent.
That is not a hot take. It is the finding of Prompting Science Report 2 from Wharton's Generative AI Labs, authored by Lennart Meincke, Ethan Mollick, Lilach Mollick and Dan Shapiro.
The phrase earned its reputation honestly. In 2022 it was a real unlock, because the models of that era did not externalise reasoning unless you asked them to.
The models changed. The habit did not.
The one-sentence definition
Chain-of-thought prompting is the technique of instructing a model to narrate its reasoning before giving a final answer, in order to improve accuracy on multi-step problems. In 2026, frontier models do this internally by default, which is why the explicit instruction has stopped paying for itself.
What does the Wharton test data actually show?
Wharton ran 198 PhD-level GPQA Diamond questions, 25 trials each, across reasoning and non-reasoning models. That is 4,950 runs per prompt per model. For reasoning models, chain-of-thought moved average accuracy by roughly 3 points up on two models and 3 points down on a third.
The specific numbers are worth knowing, because they are small and they cut both ways.
--- o3-mini: average accuracy improved 2.9 points with "think step by step" (p = 0.024)
--- o4-mini: improved 3.1 points (p = 0.003)
--- Gemini Flash 2.5: declined 3.3 points (p = 0.005)
--- Latency cost on reasoning models: 20 to 80 percent longer, or 10 to 20 extra seconds per question
The more uncomfortable result sits in the non-reasoning models, where chain-of-thought raised the average but wrecked reliability. Wharton scored a separate metric called "100% Correct", meaning the model got the same question right in all 25 trials.
On that reliability metric, Gemini Pro 1.5 dropped 17.2 points with chain-of-thought and Gemini Flash 2.0 dropped 13.1 points. Average score up, consistency down.
If your complaint about AI is "it was great yesterday and useless today", that trade is exactly the shape of your problem.
What replaced chain-of-thought in 2026?
Reasoning effort replaced it. Instead of scripting the thinking in words, you now set how hard the model should think as a parameter, then write a short prompt that names the output you want. Google, OpenAI and Anthropic all expose some version of this control.
Google says it in its own migration notes. The Gemini 3 developer guide, last updated 18 August 2026, tells developers directly: if you were previously using complex prompt engineering like chain of thought to force Gemini 2.5 to reason, try Gemini 3 with thinking_level set to high and simplified prompts.
The same page lists four levels: minimal, low, medium and high. Gemini 3.1 Pro and Gemini 3 Flash default to high.
Google also notes that OpenAI's reasoning_effort parameter maps automatically to Gemini's thinking_level through the compatibility layer, so the concept travels across vendors.
What this means if you never touch an API
You still get the control, it is just wearing a different hat. In the consumer apps it appears as a model or mode picker: a thinking or extended-thinking toggle, a reasoning model in the model list, a "fast" versus "careful" choice. Selecting the mode is the modern equivalent of typing "think step by step", and it is free of the token cost.
What does a good reasoning-model prompt look like?
A good 2026 prompt is short, states the task first, defines the finished artefact, names the constraints, and says nothing about how to think. Google's own guidance for Gemini 3 warns that the model may over-analyse verbose prompt engineering built for older models. Brevity is now a performance feature, not laziness.
Here is a template that works across Claude, ChatGPT and Gemini in their reasoning modes. Copy it and swap the bracketed parts.
Try this prompt
--- TASK: [one sentence naming the single outcome you want]
--- INPUT: [paste the source material here, or say "see the attached file"]
--- OUTPUT FORMAT: [exact shape: "a 6-row table with columns X, Y, Z", "3 paragraphs of 60 words each", "a JSON object with these keys"]
--- MUST INCLUDE: [the two or three things that make this usable, e.g. "a HK dollar figure for each row", "the source line number for each claim"]
--- MUST NOT: [the failure you keep seeing, e.g. "do not invent figures not present in the input", "do not add a summary paragraph"]
--- SUCCESS TEST: [how you will judge it, e.g. "I can paste this into the deck without editing"]
Notice what is absent. No "you are a world-class expert". No "take a deep breath". No "think step by step". No flattery, no role-play scaffolding, no instruction about reasoning.
What replaced all of that is the SUCCESS TEST line. Naming the test converts a vague request into a checkable one, and reasoning models are unusually good at optimising against a stated test.
One more small thing from Google's guide that changes results more than people expect: when you paste a long document, put your instructions after the data, not before it, and open your question with a phrase like "Based on the preceding information".
How do you apply this to real work?
Take a common task: turning a 40-page quarterly report into a client-facing summary. The old prompt would stack a persona, a step-by-step instruction and a politeness ritual. The new prompt states the artefact and the test, and lets the reasoning mode do the thinking.
The old version of that prompt usually reads something like "You are an expert business analyst. Read the attached report carefully. Think step by step. Then write a summary." It produces a competent, shapeless block of text you then rewrite.
The 2026 version, using the template above, looks like this.
--- TASK: Write the client-facing summary section of our quarterly review deck.
--- INPUT: the attached 40-page Q2 report.
--- OUTPUT FORMAT: 4 bullet points, maximum 25 words each, then one 60-word paragraph naming the single biggest risk.
--- MUST INCLUDE: a figure with its unit for every bullet, and the page number it came from.
--- MUST NOT: use any figure that does not appear in the report; do not soften the risk paragraph.
--- SUCCESS TEST: I can read the four bullets aloud to the client without checking anything.
The page-number requirement is the quiet hero here. It makes the output auditable in 30 seconds, which is what actually determines whether you trust the tool enough to use it weekly.
For a different flavour of reliability work, our earlier piece on chain-of-verification prompting covers the second pass that catches invented facts.
Where does this advice break down?
It breaks down on older and smaller models, on tasks where you need to see the reasoning, and on anything where the model must show its work to a human reviewer. Wharton found that chain-of-thought still lifts average performance on non-reasoning models, which means the advice is model-dependent rather than universal.
Three honest caveats before you delete the phrase from every saved prompt.
Small and cheap models still want the scaffolding. If you are running a mini or flash-lite tier model to keep costs down, or a local open-weight model on your own machine, chain-of-thought is often still worth its tokens. Wharton measured gains of 11 to 14 points in average accuracy on older non-reasoning models.
Sometimes the reasoning is the deliverable. If you are reviewing an argument, teaching a method, or need to check where the model went wrong, asking for the steps is legitimate. You are no longer prompting for accuracy, you are prompting for an artefact, and that is a different job.
Do not swing to the opposite extreme. Wharton also found that forcing a model to reply with only the answer and nothing else likely harms non-reasoning models, because it blocks the reasoning they would have done on their own. "Answer in one word" is a real cost, not a free efficiency.
And one Hong Kong-specific note. Mixed Cantonese and English prompts still work fine, but the short-prompt principle makes the language mix matter less than the structure. A clean bilingual template beats an elegant monolingual paragraph.
What should you do in the next 20 minutes?
Run a controlled comparison on one task you repeat weekly. Take your current best prompt, make a second version with all the reasoning instructions and persona lines stripped out, run each three times in a reasoning mode, and compare the outputs against your own success test. Three runs each is enough to see the pattern.
Score them on two axes, not one. First, is the best output better? Second, and more important, is the worst output better? Consistency is the thing that decides whether a prompt survives contact with a deadline.
Most practitioners find the stripped version wins on the second axis even when it ties on the first. That is the whole argument in one experiment.
The broader lesson is that prompting advice has a shelf life. Techniques that were essential in 2022 became optional in 2024 and became a tax in 2026, and nobody sent a memo. The only durable skill is testing your own prompts against your own work, on the model you actually pay for.
We understand AI. We understand you better. With UD by your side, AI doesn't feel cold.
Reviewed by the UD AI team.
Find Out Which Model Deserves Your Prompt
Knowing how to prompt a reasoning model is half the job. Knowing which model to point it at is the other half. UD's AI Rank puts the current models side by side on real business tasks, and if you want the technique built into a workflow that runs the same way every week, we'll walk you through every step, from model choice to prompt templates to deployment.
