Why DeepSeek-R1's Reasoning Trace Varies Run to Run

Send the same prompt to a reasoning model twice and you get two different traces — often different in length by a factor of two, and occasionally reaching the same answer by an entirely different route. Nothing is wrong. Two independent mechanisms guarantee it.

The reasoning phase is sampled

A model does not choose a token. It produces a score for every token in its vocabulary and a sampler outside the model draws one from that distribution. Reasoning is not exempt from this: the trace is generated by the same next-token loop as the answer, through the same sampler, with the same randomness.

What makes the trace vary so much more visibly than an answer is compounding. Every sampled token conditions everything after it. Choose “First, consider the constraints” over “Let me try a small case” at token forty and the next eight hundred tokens explore a different approach — not a paraphrase of the same one. Two thousand sequential draws from a distribution do not produce two thousand small differences; they produce one large one.

That is also why the answers agree more often than the traces do. The trace is a search; the answer is what the search converged on. Two different searches over the same problem frequently land in the same place, which is the entire premise of the design.

You cannot turn the sampling off

On an ordinary chat model the remedy is temperature: 0, which collapses the distribution onto its highest-scoring token and makes generation effectively greedy. That remedy is not available here. DeepSeek documents that temperature, top_p, presence_penalty and frequency_penalty are accepted by the reasoning endpoint without error and do not take effect — the fix page for that symptom covers which parameters error instead of being ignored.

The reason is that reasoning behaviour is sensitive to sampling settings in a way plain generation is not. The R1 model card recommends a temperature range for the open weights rather than a default, and explicitly warns that values outside it produce repetition or incoherence. Greedy decoding sits outside that range at the bottom: a reasoning model at temperature 0 is prone to getting stuck in loops, restating the same step, and failing to abandon a line of attack that is not working. The randomness is doing work, not adding noise.

So the API fixes the sampling settings on the reasoning path rather than letting callers choose a setting that degrades the model. That is a defensible product decision and it does mean determinism is not on offer.

Even greedy decoding would vary

Suppose the sampling were under your control and you set it to greedy. You would still not get identical output run to run, and the reason is worth knowing because it applies to every hosted model, not just this one.

Floating-point addition is not associative: summing the same numbers in a different order gives slightly different results. A serving system batches your request with whatever other requests arrived at the same moment, and the batch composition changes the shapes of the matrix operations, which changes reduction order, which changes the last bits of the logits. Almost always this is invisible. Occasionally two tokens are so close in score that the perturbation flips which one is highest, and from that point the outputs diverge completely.

A mixture-of-experts model has an additional source of the same effect. Routing decisions are made per token from scores that are themselves subject to this jitter, so a borderline routing decision can send a token to a different expert. The routing mechanism is described separately; the point here is that sparse models have one more place for a tie to break differently.

This is why a seed parameter, where a provider offers one, is documented as a best-effort aid to reproducibility rather than a guarantee. It fixes the sampler’s randomness. It cannot fix the arithmetic underneath it.

Why length varies most of all

Trace length is the most variable quantity in the response, and it is the one with a direct cost consequence. The model decides to stop thinking when its own generation reaches the point of transitioning to an answer, and that decision is itself a sampled token. An early transition ends the trace; a late one buys another few hundred tokens of exploration, each of which creates another opportunity to continue.

The distribution of trace lengths therefore has a long right tail rather than being tightly clustered around a mean. For capacity and cost planning that matters enormously: budget from a high percentile of observed reasoning_tokens, not from the average, or your max_tokens will be adequate for most requests and cut off the hard ones — which are exactly the requests you sent to a reasoning model. The billing page covers where that count is reported.

Latency inherits the same variance directly, because generation time is roughly linear in tokens produced. A reasoning endpoint has a fundamentally wider latency distribution than a chat endpoint, and a timeout set from a median will fire on the requests that most needed the model.

Working with it instead of against it

• Test on the answer, never on the trace. Any assertion over reasoning_content — a snapshot, a substring, a length bound — is a test that fails for no reason. Assert on content, and prefer semantic checks to exact matches.
• Sample repeatedly where correctness matters. Variation is exploitable: run the same prompt several times and take the majority answer. The published evaluations of reasoning models average over multiple runs for exactly this reason, and the technique transfers directly to production for high-value decisions.
• Do not cache on the trace. Key any cache on the input and the final answer. A trace is not a stable artefact and storing it as though it were produces a cache that never hits.
• Budget from percentiles. p95 of reasoning_tokens for max_tokens, p99 of latency for timeouts. Means are the wrong statistic for a long-tailed distribution.
• If you need determinism, do not use a reasoning model. A chat model at temperature 0 is far closer to reproducible, and for work that must be auditable that property may be worth more than the reasoning is.

Testing against a model that will not repeat itself

A test suite that calls a reasoning model live is a test suite that fails at random, and the usual response — rerunning until it passes — destroys its value entirely. The way out is to separate the two things you are testing, because they need completely different treatment.

• Your code around the model: record and replay. Capture real responses once, store them as fixtures, and run the suite against those. Parsing, error branches, tool loops, truncation handling and streaming accumulation are all deterministic given a fixed response, and they are where the bugs actually are. This suite should be fast, offline and never flaky.
• The model’s behaviour: an evaluation, not a test. Run a set of cases against the live model on a schedule rather than on every commit, score them, and track the score over time. The pass criterion is a rate — “at least 90% of these produce a valid answer” — because a single run of a single case is not evidence of anything.
• Assert properties, not strings. “Parses as JSON with these keys”, “the number is within tolerance of the known answer”, “the answer is in Dutch”. Exact output matching is the assertion that guarantees flakiness, and it almost never encodes what you actually care about.
• Keep fixtures honest. Recorded responses go stale as the model behind the name changes. Re-record deliberately and periodically, and read the diff — a fixture refresh that changes the shape of what comes back is telling you something a passing suite would have hidden.

The division holds for every non-deterministic dependency and it is worth stating plainly: you can test your code deterministically or you can test the model statistically, but attempting both in the same suite gives you neither.