Reasoning LLMs are ordinary transformer language models trained to spend extra compute "thinking" in tokens before they commit to an answer.

The core recipe

Models like OpenAI's o-series and DeepSeek-R1 generate a long chain of thought before the final answer, and are trained with reinforcement learning that rewards verifiably correct outcomes - a math answer that checks out, code that passes tests. Strategies like decomposing problems, self-checking, backtracking, and trying alternative approaches are not hand-coded; they emerge because they raise the reward 1DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningDeepSeek-R1 shows that large-scale RL with simple verifiable rewards elicits chain-of-thought reasoning, self-verification, and reflection without supervised reasoning traces.arxiv2Learning to reason with LLMsOpenAI's o1 is trained with reinforcement learning to produce a long internal chain of thought before answering, and accuracy scales with both train-time and test-time compute.openai. DeepSeek-R1 demonstrated this works even without any supervised reasoning traces to start from, and that the resulting behavior can be distilled into much smaller dense models 1DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningDeepSeek-R1 shows that large-scale RL with simple verifiable rewards elicits chain-of-thought reasoning, self-verification, and reflection without supervised reasoning traces.arxiv4Open-R1: a fully open reproduction of DeepSeek-R1Hugging Face's Open-R1 replicates the R1 recipe in the open: GRPO-style RL against verifiable math and code rewards, plus distillation of reasoning traces into small models.huggingface.

Test-time compute

The second ingredient is inference-time scaling: accuracy on hard problems keeps improving as the model is allowed more thinking tokens. That is why reasoning products expose effort or budget controls - Claude's extended thinking, for example, streams a visible reasoning block whose token budget the developer sets, trading latency and cost for accuracy 2Learning to reason with LLMsOpenAI's o1 is trained with reinforcement learning to produce a long internal chain of thought before answering, and accuracy scales with both train-time and test-time compute.openai3Claude's extended thinkingExtended thinking gives the model a visible scratchpad: it streams its reasoning before the final answer, and a thinking budget trades latency and cost for accuracy.anthropic.

How to think about the landscape

In practice there are four ways to get a reasoning model: pure inference-time scaling (prompting and sampling tricks), pure RL, RL on top of supervised fine-tuning (the strongest production recipe), and distillation from a bigger reasoner 5Understanding reasoning LLMsA practitioner's taxonomy of the four ways to build reasoning models: inference-time scaling, pure RL, RL plus SFT, and distillation - and when each is worth the cost.sebastianraschka. The common thread is simple: the chain of thought acts as working memory, letting a fixed-size network trade sequential token generation for effectively deeper computation 2Learning to reason with LLMsOpenAI's o1 is trained with reinforcement learning to produce a long internal chain of thought before answering, and accuracy scales with both train-time and test-time compute.openai5Understanding reasoning LLMsA practitioner's taxonomy of the four ways to build reasoning models: inference-time scaling, pure RL, RL plus SFT, and distillation - and when each is worth the cost.sebastianraschka.