When a Reasoning Model Is a Waste of Money
5 min read · updated August 3, 2026
Thinking tokens buy you one thing: more deliberation before an answer is committed to. If deliberation was not what stood between your system and a correct answer, you are paying five to twenty times the output rate for a longer wait and the same result.
The question behind all the others
Before any taxonomy, one question does most of the work: would a competent human doing this task need scratch paper? Not need to be careful — need to write something down, hold an intermediate result, try a branch and back out of it. If the honest answer is no, extra thinking has nothing to do.
There is a second filter that catches most of what the first misses: could the answer be wrong in a way you would not notice? If yes, and if the cost of not noticing is material, that changes the calculation even for a task with no deliberation in it — not because thinking makes the answer more likely to be right, but because it tends to make near-misses visible. A reasoning model asked to extract a payment date from an ambiguous contract is more likely to flag the ambiguity than to silently pick one. That is a real benefit and it is a different benefit from accuracy; price it separately.
It is a surprisingly reliable filter because it separates two things that get conflated constantly: tasks that are hard and tasks that are multi-step. Naming the histological subtype of a tumour from a description is hard and needs no scratch paper — it is a knowledge lookup, and a model that does not know will think for nine thousand tokens and then confidently not know. Balancing a chemical equation is easy and needs scratch paper, and that is where the thinking tokens go to work.
Where it changes nothing
| Task shape | Description |
|---|---|
| extraction | Pulling fields out of a document you supplied. The answer is present in the context; there is nothing to derive. Thinking adds latency and occasionally invents a reconciliation between two fields that did not need reconciling. |
| closed-set classification | Routing a ticket into one of eight queues, sentiment, language detection. The output space is tiny and the decision is a single judgement. High volume makes the cost multiplier hurt most exactly here. |
| reformatting | JSON reshaping, Markdown to HTML, CSV to objects. Deterministic transformation. If it is failing, the fix is a schema or a parser, not deliberation. |
| style and tone | Rewriting for register, translating, shortening. There is no correct answer to converge on, so there is nothing for extra steps to converge to. |
| single-document summary | Compression, not inference. Reasoning models tend to produce longer, more analytical summaries here, which readers often like less. |
| recall questions | Facts the model either has or does not. Thinking cannot retrieve what is not in the weights; retrieval augmentation can. |
There is also a whole category defined by the clock rather than the task: anything with a latency budget under a couple of seconds. Autocomplete, inline suggestions, a search-as-you-type ranker, anything in a request path a user is blocked on. A reasoning model cannot meet those budgets at all, for the reasons in latency budgets for reasoning models, so the deliberation question never arises.
Where it pays for itself
The mirror image. Every item here has interacting constraints, a checkable answer, or both.
- Constraint satisfaction. Rotas, seating plans, scheduling under conflicting rules, packing problems small enough to state in a prompt. Naive generation violates constraint four while fixing constraint two; a trace can check and revise.
- Multi-step derivation. Anything where an intermediate quantity feeds the next line — unit conversions in chains, financial models, physical estimates.
- Debugging with evidence. A stack trace, a failing test and a diff. Several hypotheses to hold and eliminate, and a cheap external check on whether the answer worked.
- Ambiguous specifications. Tasks where the right first move is enumerating cases rather than answering. Reasoning models are noticeably better at noticing that a question has two readings.
- Anything you can grade automatically. If a unit test, a schema validator or a solver can check the output, you are in the regime where test-time compute has the strongest published support — see reasoning by domain.
The error-triage test
If the taxonomy leaves you undecided, stop guessing and read your failures. Take thirty wrong answers from your current non-reasoning setup and sort each into one of four bins:
| Failure bin | Description |
|---|---|
| didn't know | A fact was absent or wrong. Fix with retrieval or a better base model. Thinking will not help; it will produce a longer wrong answer. |
| didn't look | The answer was in the context and was missed. Fix with prompt structure, chunking or attention to position. Thinking helps only incidentally. |
| lost the thread | Correct start, dropped a constraint or an intermediate value halfway. This is the bin reasoning models are for. |
| wrong shape | Right content, invalid JSON or missing field. Fix with structured output or a repair pass. Thinking is an expensive way to get a comma right. |
The decision rule is blunt and works: if fewer than about a third of your failures land in lost the thread, a reasoning model will not move your top-line number enough to justify the multiplier, and the money is better spent on retrieval or on the prompt.
The exercise is also worth doing for its own sake. Teams reach for a more expensive model when they are unhappy with quality and have not looked closely at what the unhappiness consists of — and a bin count on thirty examples usually takes under an hour and frequently redirects the whole effort. The most common outcome is that the largest bin is didn’t look, which is a retrieval or prompt-structure problem that no model upgrade fixes and that costs nothing per request to address.
The answer is usually not global
Almost nobody has a workload that is uniformly one kind of task. A support product classifies, extracts, summarises and occasionally has to reason through a billing dispute with four overlapping rules. Choosing one model for all of it means either paying reasoning prices to classify or failing the disputes.
The same holds inside a single request once it is decomposed. A retrieval-augmented answer typically involves rewriting the query, selecting passages, synthesising and formatting — four steps of which at most one benefits from deliberation. Applying a reasoning model to the whole pipeline because one stage needed it is the per-request version of the same mistake, and it is more common, because pipelines tend to be configured with a single model constant at the top of the file and nobody revisits it per stage.
Which is why the useful version of this decision is per request rather than per product, and why the next page in this cluster is about building the thing that makes it — a router that decides per request.