Pre-launch · Training and the free Foundations exam are open now. Levels 1–5 open soon. Get notified on any level page.
25 min read
Chain-of-thought prompting means asking a model to work through intermediate steps before it gives its answer. It is one of the most studied prompting techniques, and the research has moved: early results showed large gains, and later results show those gains are narrower than first thought and smaller on newer models. This module covers what the technique is, where the evidence says it helps, what it costs, and how to decide with a measurement instead of a habit.
What it is. In a chain-of-thought prompt, the model writes out steps and then the answer, all in one response. You can trigger it with an instruction ("work through the steps, then give the final answer") or by including examples that show worked steps. This is different from the staged chains in the Decomposition module. A staged chain is several prompts, with an output you inspect between each. Chain-of-thought is one prompt and one response.
The original result. Wei and colleagues (2022) gave a large model eight examples that showed worked steps before each answer. On GSM8K, a set of grade-school math word problems, accuracy for the PaLM 540B model rose from 17.9% with standard prompting to 56.9%. That finding made the technique standard practice.
Where it helps. Sprague and colleagues (2025) reviewed over 100 papers and ran their own tests on 20 datasets across 14 models. They found that the benefit shows up mainly on math, logic, and symbolic tasks, with little gain on other kinds of task. A multi-step calculation is a good candidate. A tone rewrite or a simple lookup is not.
What changed with newer models. Meincke and colleagues (2025) tested the technique on 198 difficult science questions, running each one 25 times per condition. For models that don't reason by default, it gave a small average gain but also made results less consistent, sometimes causing errors on questions the model otherwise got right. For models with built-in reasoning, the gain was marginal at best, and responses took more time and more tokens. Many current tools already work through steps without being asked.
Effect on made-up claims. A 2025 survey by Anh-Hoang, Tran, and Nguyen found that structured prompting, chain-of-thought included, reduced hallucinations in cases where the prompt was the cause. It did not remove errors that come from the model's own limits. Stepwise prompting lowers one kind of risk. It is not a substitute for checking against a source.
The steps are output, not a window. The reasoning a model writes is generated text. It can read well and still not match how the answer was produced, and a wrong answer can arrive with tidy steps. So ask for work you can check: the inputs used, each calculation, the rule applied. Then check it. Don't treat the presence of steps as evidence that the answer is right.
Decide with a measurement. Run your test set with and without the stepwise instruction, several times each. Compare accuracy, consistency across runs, response time, and length. Keep the instruction only if it earns its cost on your task and your model. Recheck when the model changes.
Gareth Lowe works in customer support at a gym chain. Members who cancel mid-month get a prorated refund, less a fee, and agents keep getting the amount wrong by hand. He wants a prompt that drafts the calculation for an agent to confirm.
Before:
A member on the $54/month plan cancels on day 19 of a 30-day
billing period. There's a $10 cancellation fee. How much is the
refund? Think step by step.
The stepwise instruction is there, but the rules aren't. The prompt doesn't say whether day 19 counts as used, how to round, or whether the fee comes out before or after proration. The model will choose, and its steps will make the choice look authoritative.
After:
Calculate a prorated membership refund. An agent will check your
work before anything is sent.
Rules:
- Unused days = days in billing period minus the cancellation
day. The cancellation day counts as used.
- Daily rate = monthly price / days in billing period. Do not
round the daily rate.
- Prorated amount = daily rate x unused days, rounded to the
nearest cent.
- Refund = prorated amount minus the cancellation fee. If the
result is below $0.00, the refund is $0.00.
Inputs:
- Monthly price: $54.00
- Days in billing period: 30
- Cancellation day: 19
- Cancellation fee: $10.00
Show each of the four calculations on its own line with the
numbers used. Then give the result on a final line in this form:
REFUND: $X.XX
If an input is missing or doesn't make sense, say which one and
stop. Don't estimate.
The agent can now check four lines: 11 unused days, a daily rate of $1.80, a prorated amount of $19.80, and a refund of $9.80. Gareth runs 20 past cancellations through the prompt with and without the "show each calculation" instruction, three times each, and compares the results with the amounts finance actually paid.
Why this is stronger. The rules that decide the answer are stated, so the steps apply Gareth's policy and not the model's guess. Each step is a line a person can verify with a calculator. The final line has a fixed form, so it is easy to find and compare. And the decision to keep the stepwise instruction rests on 20 real cases, not on the assumption that it always helps.
Setup. Pick a task with a calculation or a rule to apply: splitting a bill with tax and tip, working out a late fee, checking whether an order qualifies for free shipping. Write down five cases and work out the right answers yourself first.
Task. Write one prompt that states the rules and asks only for the answer. Write a second that also asks for each step on its own line. Run all five cases through both, twice each.
What a good result looks like. You have a small table of right and wrong answers for each version. You can say whether the steps changed accuracy, whether results varied between runs, and how much longer the responses were. If a step was wrong, you can point to the line.