Skip to content
Multigrid

Reasoning models & test-time compute

What changes when a model spends tokens thinking before it answers — in the bill, in the latency, and in the set of tasks that actually get better.

For most of the last decade, making a model better meant making it bigger and training it longer. Test-time compute is the other lever: keep the weights fixed and spend more work per question. That is what a reasoning model does, and it is also what best-of-N sampling, tree search and a verifier reranking candidates do. They are the same trade wearing different clothes.

The trade is not free and it is not uniform. It buys a lot on problems where a wrong answer is cheap to recognise, very little where it is not, and it can lose outright on problems simple enough that a first instinct was already correct. These pages are about telling those cases apart before you have paid to find out — and about the mechanics that make reasoning models awkward to operate: hidden tokens you are billed for, a time-to-first-token that no longer predicts anything, and a budget knob that is a hint rather than a limit.

What Is a Reasoning Model? A Definition You Can Check

Defines a reasoning model by three checkable properties of its billing and its response object, rather than by what the vendor says it does.

5 min read

Test-Time Compute: Paying for Thinking Instead of Parameters

The trade behind reasoning models — fixed weights, variable work per question — and what moving cost from training to inference changes about your economics.

5 min read

Reasoning Effort Settings: What the Knob Actually Does

What an effort level or thinking budget changes in the request, the response and the invoice — and a three-run procedure for calibrating it on your own evaluation set.

5 min read

When a Reasoning Model Is a Waste of Money

A taxonomy of task shapes where extra thinking changes the answer not at all, and the four questions that separate them from the ones where it pays.

5 min read

Hidden Reasoning Tokens: Billed, Invisible, and Yours to Handle

The three independent choices a provider makes about a reasoning trace — return it, bill it, carry it — and how to tell which ones apply to you from a response you already have.

5 min read

Reasoning Models vs Chain-of-Thought Prompting

The structural differences between a trained reasoning policy and a prompt that asks for working — who controls the trace, what gets billed, what can be cached — and when each is the right tool.

5 min read

Latency Budgets for Reasoning Models

Why time to first token stops predicting anything once a model thinks first, and how to set timeouts, infrastructure limits and user expectations around a variable-length silence.

5 min read

Can You Trust a Model’s Stated Reasoning?

What the faithfulness literature actually tested, what it found, and what follows for anyone treating a visible chain of thought as an explanation.

5 min read

Overthinking: When More Reasoning Makes Accuracy Worse

The published overthinking results, the task shapes where extra deliberation is known to hurt, and how to detect the effect on your own evaluation set.

5 min read

Routing Between a Reasoning Model and a Fast Model

Three ways to decide per request which model handles it — heuristics, a classifier, and escalation after failure — with the code and the arithmetic for each.

5 min read

Self-Verification: What Asking a Model to Check Its Work Catches

The published limits of intrinsic self-correction, the specific error classes a critique pass does catch, and the design rule that separates the two.

5 min read

Process Supervision vs Outcome Supervision

Why grading each step of a solution rather than only its final answer changed how reasoning models are trained, and what it costs to do.

5 min read

Monte Carlo Tree Search for LLM Reasoning

How tree search is applied to reasoning steps rather than game moves, what each of the four phases becomes, and why the token arithmetic keeps it out of most production systems.

5 min read

Best-of-N Sampling: The Simplest Test-Time Scaling There Is

The arithmetic behind the coverage curve, why real curves flatten sooner than the formula predicts, and why the picker matters more than N.

5 min read

Verifiers and Reward Models at Inference Time

Using a cheap checker to pick among expensive candidates — which kinds of verifier exist, how good yours has to be to help, and how to keep it from being gamed.

5 min read

Reasoning on Maths vs Code vs Text: Why the Gains Differ

Why extra thinking transforms results on maths and code and does much less for open-ended text — an argument about verifiability rather than about difficulty.

5 min read

The Cost Curve of Reasoning: Working Out Accuracy per Dollar

A cost model you fill in from your own evaluation and your own rates, with the break-even arithmetic that says whether a more expensive tier is worth buying.

5 min read

Streaming Partial Thoughts: UX for a Model That Thinks for 40 Seconds

The stream events that carry a reasoning trace, and the interface patterns that make a long silence tolerable without pretending the trace is the answer.

5 min read

Distilling a Reasoning Model Into a Small One

How reasoning capability is transferred into small models by supervised training on generated traces, what the published results showed, and what you can and cannot do with the technique yourself.

5 min read

Do Reasoning Models Actually Reason? The Honest Answer

The evidence on both sides of the argument, why the two camps keep talking past each other, and an operational definition you can test on your own task instead.

5 min read

Other topics

Reasoning models & test-time compute · Multigrid