Reasoning model tokens: when the extra thinking actually pays off
What a reasoning mode actually does
When you flip on “extended thinking” or set a reasoning_effort parameter, the model doesn’t get smarter. It gets more tokens to work with before it has to commit to an answer. Instead of jumping straight from your prompt to the final response, it generates a block of intermediate text first, a scratchpad where it can lay out steps, check its own arithmetic, consider an alternative approach, and back out of a wrong turn before you ever see it.
That scratchpad is not free. It’s generated the same way any other output token is generated: one at a time, autoregressively, billed the same way as the tokens in your final answer. A reasoning trace that runs three thousand tokens before the model writes a two-hundred-token reply means you paid for those three thousand tokens even though you never read most of them. Some providers hide the raw trace and show you a summary instead, but the meter is still running on the full thing behind the scenes.
This is the part that gets glossed over in marketing copy: reasoning mode is not a quality toggle, it’s a compute toggle. You’re buying the model more attempts at self-correction. Whether that purchase is worth it depends entirely on whether the task has room for self-correction to matter.
Where the extra tokens actually help
The clearest wins show up on tasks with a verifiable internal structure, places where the model can check its own work mid-stream and catch a mistake before it ships.
Multi-step arithmetic and unit conversions are the obvious case. A model reasoning through “convert this to metric, then apply the tax rate, then split three ways” has natural checkpoints where it can re-verify a number before using it in the next step. Without that space, an error in step one propagates silently through steps two and three.
Debugging is the case I lean on most in day to day work. Handing a model a stack trace plus the surrounding function and asking “why does this throw” is a fundamentally different task from asking it to write a new function from scratch. Diagnosis is a search problem: the model has to consider several candidate causes, rule them out against the evidence in the trace, and only then commit to one. That elimination process is exactly what a reasoning trace is for. Skip it and you get a plausible-sounding guess based on pattern matching against similar bugs it’s seen, which is often right but wrong often enough to waste your afternoon when it isn’t.
Planning and constraint satisfaction are the third bucket: scheduling problems, “arrange these tasks given these dependencies,” multi-file refactors where changing one function signature has to be traced through every caller. These tasks have a combinatorial structure where a single pass through the prompt genuinely isn’t enough context to hold in one shot. The extra tokens let the model build up an internal map before answering.
The common thread: these are all tasks where the correct answer isn’t obvious from the prompt’s surface form, and getting there requires the model to explore, check, and sometimes reverse itself.
Where the extra tokens are just a slower, pricier answer
The failure mode I see most often on my own API bills is turning on reasoning for tasks that have no internal structure to reason about.
Formatting and rewriting tasks are the biggest offender. “Convert this JSON to YAML,” “shorten this paragraph,” “fix the grammar in this email.” There’s no hidden multi-step logic here. The model either knows the transformation or it doesn’t, and giving it three thousand tokens of scratchpad space doesn’t change what it knows. You just paid for the model to think out loud about a task that has no thinking to do.
Simple factual retrieval is the same story. “What’s the capital of this country,” “what does this function name mean,” “summarize this paragraph.” If the answer is sitting right there in the model’s weights or in the context you gave it, a reasoning pass doesn’t improve the retrieval, it just adds latency and tokens on top of a lookup.
Style and tone tasks fall into this bucket too. Asking a model to write in a specific voice or match a template is a pattern-matching task, not a logic task. Reasoning mode tends to produce the same output as a direct call here, just slower and at a higher token count, because there’s nothing to verify against.
The tell, in my experience, is asking yourself: if I made a mistake on step one of this task, would I notice by step three? If there’s no step three, there’s nothing for the extra tokens to catch.
Setting the dial instead of guessing
Every major reasoning API gives you a way to control how much thinking budget the model gets, rather than a flat on-off switch, and that control is worth using deliberately instead of leaving on a default.
OpenAI’s reasoning models expose a reasoning_effort setting (low, medium, high) that scales how long the model is allowed to deliberate before answering. Anthropic’s extended thinking lets you set a token budget directly, so you can cap a reasoning pass at, say, a couple thousand tokens for a routine debugging call and open it up wider for something gnarly. Gemini’s thinking models work on a similar budget concept.
The practical move is to route by task type rather than by vibe. I keep reasoning off by default for anything that’s formatting, extraction, or short rewriting, because the failure cost of a wrong answer is low and easy to spot, and the token cost of reasoning isn’t worth paying on every call. I turn it on, and turn the budget up, for debugging sessions, anything involving multi-step math, and any prompt where I’m asking the model to plan out a sequence of edits across more than one file. For everything in between, a low or medium effort setting is usually enough headroom to catch the kind of one-step error that a zero-reasoning call would sail past, without paying for the kind of exhaustive search that a genuinely hard problem needs.
If you’re building on top of these APIs rather than just chatting with them, this routing decision belongs in your code, not in a system prompt. A support-ticket classifier and a code-review agent sitting behind the same product should not share a reasoning setting, because they’re paying for fundamentally different amounts of internal search.
The bill you’re actually paying
The uncomfortable part of reasoning mode is that the trace length isn’t fully predictable ahead of time. You can cap the budget, but within that cap the model decides how much of it to use, and a task that looks simple to you can sometimes trigger a long trace because the model talked itself into checking something twice. Watching your own token logs after switching a workflow to reasoning mode is the only way to know whether you’re actually getting proportionally better answers for the extra spend, or just a longer, more expensive way to arrive at the same output you’d have gotten without it.
That’s also why “always use the reasoning model” is a fine default for a chat window, where a human is watching the answer and can just wait an extra few seconds, but a bad default for a production pipeline. In a high-volume API integration, every call that didn’t need the extra tokens is quietly inflating your bill for no gain.
A simple rule to route by
Reach for reasoning mode when the task has a check-your-work step built into it: math, debugging, planning, anything where a wrong intermediate conclusion would be caught by continuing to think. Skip it for retrieval, formatting, and style tasks where there’s no intermediate step to get wrong in the first place. Set the budget deliberately per task type in your code rather than leaving one global setting, and check your actual token logs after the fact instead of assuming the reasoning trace did what you hoped.
If you want more of this kind of breakdown on the AI tools and coding assistants actually worth paying for, head back to the Ai Tool Gazette home page for the rest of our explainers.