NOTES / LLM FUNDAMENTALS

Reasoning models

Models that think before answering: chain of thought, spending compute at inference time, reinforcement learning, and how thinking is controlled and paid for.

UPDATED 18 NOV 2026#llm#reasoning#chain-of-thought#test-time-compute#thinking

Start with chain of thought

Even very large models struggled with multi-step problems like math word problems. Wei and colleagues showed that if you prompt a model to produce a chain of thought, a series of intermediate reasoning steps, performance on arithmetic, commonsense and symbolic tasks improves markedly1. With just eight exemplars, a 540B-parameter model reached state-of-the-art accuracy on the GSM8K math benchmark at the time of publication1. The reasoning ability emerged in sufficiently large models via prompting alone1.

The mechanism is mundane and powerful: each generated token is more computation. Writing out steps lets the model do work it cannot do in a single forward pass.

Spending compute at inference time

If thinking longer helps, how should you spend extra compute? Snell and colleagues studied scaling inference-time computation and found that the effectiveness of different approaches varies critically with prompt difficulty. That motivated a “compute-optimal” strategy that allocates test-time compute adaptively per prompt, improving efficiency by more than 4× over a best-of-N baseline2. In a compute-matched comparison, on problems where a smaller model already had non-trivial success, extra test-time compute could beat a model 14× larger2.

Diagram: three ways to spend extra inference compute, best of N, search and revision, with the best choice depending on prompt difficulty

Diagram: my own, summarising the strategies compared in 2.

Training models to think

Prompting for chain of thought is optional; reasoning models are trained to do it. DeepSeek reported that reasoning can be incentivised with pure reinforcement learning, without human-labelled reasoning trajectories, and that behaviours emerged during RL training3. Their final R1 model reached performance comparable to OpenAI’s o1-1217 on reasoning tasks3. Training details are in How models are trained.

How thinking shows up in an API

Anthropic’s documentation describes thinking as the model working through a problem in its own words before answering: restating the question, trying approaches, checking intermediate results, and abandoning paths that fail4.

Diagram: without thinking the model answers directly; with thinking it works through the problem in thinking tokens first

Diagram: my own, following the behaviour described in 4.

Controls have changed quickly, so check current docs:

  • The older “extended thinking” form sets a thinking token budget with budget_tokens, and Claude thinks against that budget before its final answer. Thinking tokens count toward max_tokens, so the budget must leave room for the reply4.
  • That manual mode is now deprecated. Claude Opus 4.7 and later models reject it with a 400 error, and the documented replacement is adaptive thinking with an effort setting in place of a token budget4.

When to use a reasoning model

My working guide, not from the sources:

  • Use it for multi-step maths, code with tricky logic, planning, and tasks where checking intermediate results matters.
  • Skip it for simple lookup, formatting, classification, or latency-sensitive paths. Thinking costs tokens and time.
  • Measure. Compare reasoning on and off on your eval set at several effort levels; do not assume more is better.
  • In agents, reasoning helps with planning and recovering from errors, but each turn is slower. See ReAct and planning.

Reasoning does not remove failure modes. Next: Capabilities and failure modes.

Sources

  1. [1]Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou · arXiv, NeurIPS 2022 · 2022PARAPHRASED
  2. [2]Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar · arXiv · 2024PARAPHRASED
  3. [3]DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI · arXiv · 2025PARAPHRASED
  4. [4]Extended thinking — Anthropic, Claude Platform docsPARAPHRASED

© Ashish Kumar. All rights reserved. Quoted and cited material belongs to its owners. How I attribute sources.

Product, company and framework names are trademarks of their respective owners. Mentioning them does not imply affiliation or endorsement.