What is a reasoning model?

Between your question and a reasoning model’s answer sits a chain of intermediate tokens, often thousands of them, where the model works through the problem step by step, correcting dead ends before it commits to a conclusion. This extended scratchpad is what separates a reasoning model from a standard one; the answer only appears after this internal work is done.

Where the idea came from

Chain-of-thought prompting demonstrated the mechanism. In 2022, researchers at Google showed that asking a large language model to generate intermediate reasoning steps, rather than jumping straight to an answer, substantially improved accuracy on arithmetic and symbolic reasoning tasks (Wei et al., 2022). The key ingredient was not a change to the model; it was prompt structure. Show the model examples of step-by-step working, and it produces step-by-step working.

That was a prompting trick. A reasoning model internalises it. Through reinforcement learning, the model learns that generating an extended chain of thought before answering leads to better scores on hard problems. The behavior bakes into the weights rather than depending on how you ask.

Test-time compute is the technical core

The right framing is compute, not cognition. Every model runs a fixed number of operations to generate each output token. For a standard model, the answer arrives after a few output tokens. A reasoning model first generates hundreds or thousands of thinking tokens, each a standard forward pass, before producing the final response. That extended sequence is the compute budget.

This matters because compute at inference time scales independently from model size. Research from DeepMind established that scaling test-time compute is a genuine axis, not a prompt trick, and that a smaller model given a generous thinking budget can outperform a model with many times more parameters operating at standard cost (Snell et al., 2024).

The empirical evidence is sharp. DeepSeek’s R1-Zero model, trained with reinforcement learning and no supervised data, saw its pass rate on the AIME 2024 mathematics benchmark climb from 15.6% to 71.0% Pass@1 purely through learning to generate longer reasoning chains (DeepSeek-AI, 2025). The architecture did not change; the model learned to spend more compute before answering.

What “thinking tokens” actually are

During inference, a reasoning model generates a stream of tokens that function as a scratch pad of partial solutions and self-corrections. In many interfaces this appears as a collapsible “thinking” section. Sometimes the trace is hidden entirely; the compute was still spent either way.

These tokens cost money and take time just like any output tokens. A response involving a long thinking sequence can take tens of seconds and run to several thousand tokens before the answer appears. That cost is worth paying when the problem is hard. It is not worth paying when it is not.

The non-obvious limit of the visible trace

The visible chain of thought is not a reliable record of internal computation.

Research testing chain-of-thought faithfulness found that when reasoning models used hints present in the context, they mentioned those hints in their visible reasoning in a minority of cases, typically below 20%, even when the hint clearly influenced their answer (Chen et al., 2025). The trace is scaffolding the model uses to arrive at an answer; it is not a log of how that answer was reached. A confident-looking chain of thought can precede a wrong conclusion.

This does not reduce the accuracy benefit on hard tasks. It does mean the visible reasoning is less useful for auditing a model’s logic than it first appears.

When the compute is worth spending

Reasoning models show a genuine accuracy advantage on tasks that require tracking multiple dependencies: multi-step mathematics, code with complex logic, planning under constraints, and research synthesis. The longer thinking trace produces better results on these because the hard part of the problem is the intermediate reasoning.

The advantage disappears on tasks where there is no hard intermediate reasoning to do. Summarization and conversational replies do not benefit from extended thinking chains, and generating them wastes time and money. A standard large language model handles these faster with no accuracy difference worth measuring.

The practical choice is about routing: reserve the compute for problems that actually require it.

Sources

• Wei, J. et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903, 28 Jan 2022.
• Snell, C. et al. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. arXiv:2408.03314, 6 Aug 2024.
• DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948, 22 Jan 2025.
• Chen, Y. et al. Reasoning Models Don’t Always Say What They Think. arXiv:2505.05410, 8 May 2025.