The Seed Parameter and What “Mostly Deterministic” Means

seed is the parameter people reach for when they want the same request to give the same answer twice. It helps. OpenAI documents it as best-effort rather than guaranteed, and the reasons it cannot be guaranteed are structural rather than a gap somebody will close.

What seed does

Sampling from a language model needs randomness: the model produces a distribution over the next token and something draws from it. That draw uses a pseudo-random number generator, and seed sets its starting state.

{
  "model": "gpt-4o-2024-08-06",
  "seed": 42,
  "temperature": 0.7,
  "messages": [{"role": "user", "content": "Name a colour."}]
}

Same seed, same prompt, same parameters, same model, same backend: the same sequence of draws, and therefore the same output. That is the intended behaviour and it usually happens. Note that seed only matters when there is randomness to control — at temperature: 0 the sampler takes the highest-scoring token regardless, and seed changes nothing. Seed is for reproducing a sample, not for removing sampling.

The caveat, in OpenAI’s own terms

The Chat Completions reference describes seed as a beta feature and says repeated requests with the same seed and parameters should return the same result on a best-effort basis, while directing you to monitor system_fingerprint for backend changes. Two words in that are doing the work: should and best-effort. Neither is the language of a guarantee, and OpenAI is being straightforward rather than evasive — the guarantee is not available to give.

It is also worth being clear about what is not being claimed, because the parameter attracts more expectations than it advertises. Seed does not make the model deterministic; it makes one source of randomness reproducible. It says nothing about determinism across models, across snapshots of the same model, across regions, or between the Chat Completions and Responses endpoints. And it is scoped to a single request: seeding two requests identically does not make them produce complementary or related answers, only individually reproducible ones.

It reproduced yesterday and does not today

This is the shape the problem actually arrives in, and it is worth walking because the debugging instinct it triggers is the wrong one.

You build a regression suite. Each case pins a model snapshot, sets seed, sets temperature, and asserts the output against a stored string. It passes. It passes on every run for three weeks, which is long enough for everybody to believe the suite means something. Then one morning four cases fail, the diffs are small — a rephrased clause, a different ordering in a list — and nothing in your repository changed.

The instinct is to look for a change on your side: a dependency bump, a whitespace difference in a prompt template, a stray trailing newline. It is worth eliminating those, and they are usually not it. The evidence that settles it is in the response you already have, if you logged it:

# the stored run
model: gpt-4o-2024-08-06
seed: 42
system_fingerprint: fp_4e2b1c9f      ← recorded three weeks ago

# today
model: gpt-4o-2024-08-06
seed: 42
system_fingerprint: fp_a91d7c04      ← different backend

Different fingerprint, same everything else. The backend serving that snapshot changed, your seeded run is now a seeded run on a different machine or a different inference stack, and there is no action available that restores the old outputs. Nothing was broken and nothing can be fixed; the assertion was measuring something that was never promised to hold.

The genuinely awkward version is the same failure with matching fingerprints, which happens too. Then the cause is batching or floating-point ordering, both described below, and the diagnostic ends there — there is no further field to inspect and no support ticket that resolves it. A suite that can fail for a reason nobody can act on is a suite that will eventually be ignored, which is the real cost of building on exact-match assertions.

Streaming does not change any of this, and one detail is worth knowing: system_fingerprint is repeated on every chunk of a streamed response, identical each time, so you can capture it from the first chunk without waiting for the stream to finish. The chunk anatomy is in the streaming chunk format.

Three reasons it is not enough

Floating-point addition is not associative

This is the deep one. GPU kernels sum products in whatever order the hardware schedules them, and in floating-point arithmetic (a + b) + c is not always equal to a + (b + c). The difference is in the last bits of a logit — far too small to matter, until two candidate tokens are close enough that the ordering flips. Then one token differs, and because generation is autoregressive, every subsequent token is conditioned on a different prefix. A rounding difference in the eighth decimal place becomes a different second paragraph.

Batching changes the arithmetic

Inference servers batch concurrent requests to use the hardware efficiently. The size and composition of that batch depends on what other traffic arrived in the same few milliseconds — nothing to do with you. Different batch shapes select different kernels and different reduction orders, which is the previous paragraph again, driven by a variable you cannot see or set. Your request at 03:00 and the identical one at 13:00 are not running the same computation.

The backend moves

Even with a pinned snapshot, what serves it changes: inference engine versions, quantisation, kernel libraries, hardware generations, routing between clusters. A pinned model id fixes the weights, which is a real and worthwhile guarantee — see pinning a model snapshot — but it does not fix the machine or the software stack executing them.

Mixture-of-experts routing adds a fourth mechanism where it applies: if expert selection is affected by batch composition, two identical prompts in different batches can activate different experts and produce different logits before sampling is even reached.

What system_fingerprint is actually for

Every response carries a system_fingerprint, an opaque string identifying the backend configuration that served it:

{
  "id": "chatcmpl-...",
  "model": "gpt-4o-2024-08-06",
  "system_fingerprint": "fp_4e2b1c9f",
  "choices": [ ... ]
}

It is not an input and cannot be requested. Its purpose is diagnostic: if two seeded requests differ and the fingerprints also differ, the backend changed underneath you and that is the explanation. If the fingerprints match and the outputs still differ, you have hit batching or floating-point non-determinism, and there is no parameter that fixes it.

The right use is therefore to record it alongside every response you might later need to explain, and to alert on it changing for a workload you believed was stable — a fingerprint change is advance notice that your golden outputs are about to stop matching. Its exact semantics are covered in what system_fingerprint tells you.

Building on reproducibility you can have

Before the things that work, the three workarounds that do not, because each is reached for first. Retrying until the output matches converts a correctness problem into a latency and cost problem and does not solve it: nothing guarantees the target output is still reachable at all after a backend change. Driving temperature to 0 and calling it deterministic removes the sampler but not the arithmetic — greedy decoding still picks the argmax of logits computed in a non-deterministic order, so two near-tied tokens can still swap. And pinning system_fingerprint is not possible; it is a response field, not a request parameter, and there is no way to ask for a particular backend.

The general principle: do not build anything whose correctness depends on byte-identical model output. That property is not offered, so a system that needs it is a system that will break at an unpredictable time for a reason nobody can reproduce.

1. Cache instead of re-deriving. If you need the same answer twice, store the first one. A cache keyed on the request hash gives exact reproducibility, costs nothing on the second call, and is unaffected by anything OpenAI does to its backend.
2. Test properties, not strings. An evaluation that asserts the output parses, conforms to the schema, contains the required entity and stays under a length is stable across model variation. One that asserts an exact string is measuring the weather.
3. Pin the snapshot anyway. Seed does not guarantee determinism, but an unpinned alias guarantees the opposite: the model behind gpt-4o changes without notice. Pinning removes the largest and most abrupt source of drift even though it does not remove all of them.
4. Use temperature 0 for extraction. If the task is pulling a value out of a document, sampling variety is not a feature. Greedy decoding is more stable than seeded sampling because it removes one source of variation entirely — it does not remove the floating-point one, but it is the strictly better starting point. The interaction with top_p matters here and is covered in setting temperature and top_p together.
5. Log the fingerprint with the output. When somebody asks in three months why the answer changed, the fingerprint is the only evidence that will still exist.