Reasoning models explained: when thinking tokens are worth it

A reasoning model is a large language model that’s allowed to generate a chunk of internal, step by step working before it commits to a final answer, and that internal working gets billed to you as tokens even though you usually don’t see all of it. OpenAI calls this “reasoning tokens.” Anthropic calls it “extended thinking.” Both cost real money on top of whatever answer eventually lands in your chat window or API response. If you’re picking models for a product, a research task, or a coding agent, this distinction is the difference between a call that costs a fraction of a cent and one that costs forty times more for what looks, from the outside, like the same question.

I run product and content work through several of these models every week, and the pattern I keep seeing is people either turning thinking on for everything, which burns budget on tasks that never needed it, or never turning it on, then wondering why the model keeps botching a multi-step problem a smarter setup would have caught. This is the plain-language version of what’s actually happening, so you can make that call on purpose instead of by default.

what it is

A reasoning model generates a hidden or partially visible sequence of intermediate tokens, often called a “thinking” or “reasoning” trace, before producing its final response. That trace is the model working through the problem: breaking it into steps, checking an approach, backtracking if something doesn’t add up, then writing the answer you actually see. This is different from a standard chat model, which goes straight from your prompt to its output in one pass.

OpenAI’s o1, released as a preview on September 12, 2024, was the first widely used model built this way, described in the company’s post Learning to reason with LLMs. Anthropic followed with Claude 3.7 Sonnet in February 2025, which added an “extended thinking” mode you can toggle on a per-request basis, detailed in Anthropic’s Claude 3.7 Sonnet announcement. DeepSeek published a fully open account of how it trained a reasoning model with reinforcement learning in its DeepSeek-R1 paper in January 2025, which is worth reading if you want the actual training mechanics rather than a marketing summary.

how it works

Under the hood, the model is still just predicting the next token. What’s different is that before it writes your answer, it’s allowed to spend a budget of tokens on an intermediate reasoning pass that the model itself generates and then reads back, essentially thinking out loud in text before switching to answer mode. Those intermediate tokens are computed the same way as any other output token, which is why they’re billed as output tokens on your invoice even when the interface hides most of them from you.

The vendors handle visibility differently. OpenAI’s reasoning models, per its reasoning models guide, keep the raw chain of thought hidden and show you a summarized version instead, partly to stop competitors from training on the raw traces. Anthropic’s extended thinking is more transparent: you can read the actual thinking block the model produced, and you can set a thinking token budget (for example, capping it at 8,000 or 32,000 tokens) to control how much the model is allowed to deliberate before it has to answer. Either way, more thinking tokens means more compute, more latency before you get a response, and a bigger bill, regardless of whether the final answer changes at all.

why it matters

Cost and latency are real tradeoffs, not rounding errors. OpenAI’s o1 launched at $15 per million input tokens and $60 per million output tokens, several times the rate of a standard chat model at the time, and reasoning tokens count toward that output total even when you never read them. If you route every request through a reasoning model by default, you’re paying that premium on questions a cheaper model would have answered just as well.

It genuinely improves multi-step tasks. Math proofs, debugging a failing test, planning a sequence of tool calls in an agent, or untangling a logic puzzle are exactly the cases where letting the model check its own work before answering catches mistakes a single forward pass would miss. This is where the extra spend earns its keep.

It changes how you should prompt. Reasoning models generally do better with short, direct instructions and worse with the long few-shot prompts and heavy scaffolding that helped older chat models. If you’re porting a prompt built for GPT-4o or Claude 3.5 Sonnet straight into a reasoning model, you may be fighting the model’s own process instead of helping it.

It adds a data governance question. A visible thinking trace is more raw text leaving your system, and if you’re piping it through a third-party logging or observability tool, that’s additional sensitive content sitting somewhere you don’t control. I’d treat reasoning traces the same way I’d treat any other model output that might reference user data, which is a topic theprivacywire.com covers in more depth if you’re building anything that logs LLM traffic.

common misconceptions

“More thinking tokens always means a better answer.” Not true. There’s a point past which extra deliberation stops helping and can even talk a model into second-guessing a correct first instinct on a simple question. Set a thinking budget suited to the task’s actual difficulty, not the maximum available.

“What I’m reading is the model’s real internal reasoning.” Treat displayed reasoning, especially OpenAI’s summarized version, as an artifact generated for your benefit, not a verified trace of the underlying computation. It’s useful for debugging a wrong answer, but it isn’t a guarantee of why the model actually landed where it did.

“Reasoning models should replace every other model call.” For classification, extraction, short chat replies, or anything with a narrow, well-defined answer, a standard model is faster and cheaper with no quality loss. Matching the model to the task is the actual skill here, which I get into more in how to cut your LLM API bill in half and how to choose an embedding model in 2026.

“Thinking tokens are just a pricing trick.” They correspond to genuine extra inference compute, which is why every vendor bills them as output tokens rather than throwing them in for free. The cost is real because the compute is real.

where to go from here

If this is your first pass at reasoning models, three things are worth reading next. Context windows explained: how big is big enough covers the other budget you’re managing alongside thinking tokens. How to cut your LLM API bill in half goes into routing strategies so you’re not defaulting to the expensive model for every call. How to choose an embedding model in 2026 is the companion piece for retrieval-heavy workloads, where reasoning depth matters less than search quality. And the blog index has the rest of the explainers if you want the wider picture before you start building.

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-07-23.