Overthinking: When More Reasoning Makes Accuracy Worse

More thinking is not monotonically better, and this is not folklore — it has been characterised in the literature under its own name. The useful part is knowing which task shapes it hits, because they are predictable enough to route around.

What has been documented

The term entered the literature with Chen and colleagues’ 2024 paper Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (arXiv 2412.21187). Their observation is easy to state: on genuinely trivial problems, reasoning models spend enormous numbers of tokens producing multiple redundant solution rounds, where the first round already had the answer and every subsequent one contributes nothing. The paper frames this as an efficiency problem — you are paying for rounds two through six — and proposes measuring the token cost of the rounds after the first correct one.

A second result came from a different direction. Apple’s 2025 paper The Illusion of Thinking evaluated reasoning models on controllable puzzle environments and reported three regimes: on low complexity, non-reasoning models matched or beat reasoning ones; on medium complexity, reasoning helped; and past a complexity threshold both collapsed. The finding that attracted most attention was counter-intuitive — as problems approached that threshold, the models began spending fewer thinking tokens, not more, despite having budget remaining. The paper was contested, notably by a 2025 response arguing that some of the failures were output-token limits and unsolvable puzzle instances rather than reasoning collapse, and that dispute is worth knowing about before you cite either side. The low-complexity regime, though, is not seriously disputed and is the one that shows up in production.

It is worth separating two claims that get merged when this literature is summarised. The efficiency claim — that reasoning models spend wildly disproportionate token counts on easy problems — is well-supported and directly costs you money. The accuracy claim — that more thinking makes answers worse — is narrower, task-dependent and more contested. You should act on the first unconditionally and test the second on your own data, which is what the rest of this page is for.

Five shapes that get worse

Notice how much overlap there is with the tasks where reasoning changes nothing. That is not a coincidence: the same absence of intermediate steps that makes the extra tokens useless also gives a long trace room to wander. Useless and harmful are neighbouring conditions here.

The flip-rate test

Aggregate accuracy hides this effect completely. If high effort fixes eight items and breaks six, your accuracy moved by two and you will conclude that effort barely matters — when in fact fourteen items changed answer and half of them changed for the worse. Measure the transitions, not the total.

const low  = await runEval({ effort: "low"  });   // [{ id, correct }]
const high = await runEval({ effort: "high" });

const byId = new Map(low.map(r => [r.id, r.correct]));
let fixed = 0, broken = 0;

for (const r of high) {
  const was = byId.get(r.id);
  if (!was && r.correct) fixed++;
  if (was && !r.correct) broken++;
}

console.log({ fixed, broken, net: fixed - broken });
// broken > 0 with net ~= 0 is the overthinking signature.
// Read those cases. Do not read the aggregate.

Then open the broken ones. The characteristic pattern is a trace that reaches the right answer early, expresses doubt, considers an alternative reading of the question, and commits to the alternative. You can usually tell within two examples whether that is what you have.

This is also why published benchmark tables are nearly useless for detecting the effect. A benchmark reports one aggregate per model, so a change that fixes as much as it breaks appears as a wash and the items it broke are invisible. If your product cares about not regressing answers that already worked — and most products with users do, because a regression is far more noticeable than a missing improvement — then the flip count is the number to govern by, and the aggregate is the number to report to people who are not making the decision.

The mirror failure

Overthinking has a documented twin. A 2025 paper on what its authors called underthinking in o1-like models described traces that switch between candidate approaches too frequently — abandoning a promising line after a couple of steps, then another, so that a large token budget is spent without any single approach being carried far enough to succeed.

The two failures look similar from outside (long trace, wrong answer) and want opposite fixes. Overthinking wants less budget or a lower effort level. Underthinking wants a prompt that asks the model to commit to one approach and see it through, or a best-of-N setup where the exploration happens across separate samples rather than inside one trace. Read the trace before choosing; guessing here costs you a week.

What to do about it

• Do not set effort globally. The correct level is per task type, and for a mixed workload the correct answer is a router.
• Cap the budget deliberately. A budget is also a stopping rule. If your task is bounded, giving unbounded room to deliberate is not generosity.
• Make the answer format terse. The overthinking papers describe redundant solution rounds inside the trace; constraining the visible answer will not stop them, but it does stop the second, cheaper version of the same problem, where the answer re-litigates the trace.
• Keep a non-reasoning arm in your eval forever. It is the only way you will notice that the expensive path has quietly become the worse one after a model update.

A closing point on framing, because it affects how this gets received internally. Overthinking is not evidence that reasoning models are overrated; it is evidence that effort is a parameter with an optimum rather than a quality setting with a maximum. Nobody would run a database with every index enabled on the grounds that indexes are good. The same discipline applies here, and the flip-rate test is how you find the optimum instead of assuming it sits at the top of the range.