The random_seed Parameter in the Mistral API

Mistral’s random_seed takes an integer and makes sampling reproducible. It does not make the API reproducible, and the gap between those two statements is where a day gets lost.

The parameter

It is a top-level field on the chat completions request, an optional integer, defaulting to unset. Mistral names it random_seed — not seed, which is what the OpenAI-shaped schema calls the equivalent field. If you are porting a request body across, this is one of the handful of names that differs, and an unrecognised extra field is silently ignored rather than rejected by most clients, so the symptom is “my seed does nothing” rather than an error.

curl -s https://api.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-small-latest",
    "messages": [
      {"role": "user", "content": "Name three cities in France."}
    ],
    "temperature": 0.7,
    "random_seed": 42
  }'

Send that request twice, unchanged, and you should get the same completion both times far more often than you would without the seed. Change random_seed to 43 and leave everything else alone, and you get a different draw from the same distribution — which is the useful part, because it gives you variation you can label rather than variation you cannot reproduce.

What the seed actually controls

A model does not choose a token. It emits a score for every token in its vocabulary, and a sampler picks one using a pseudorandom number generator. The seed initialises that generator. Two runs with the same seed, the same logits and the same sampling parameters walk the same sequence of random draws and therefore select the same tokens.

Everything in that sentence is a condition. The seed fixes the sampler’s randomness and nothing else. It has no influence on the logits themselves, on which model version served the request, or on anything that happens between your prompt and those logits being computed. If any of those differ, the sampler is drawing from a different distribution and identical seeds produce different text with no contradiction.

Why the same seed still drifts

Mistral documents reproducibility from a seed as best-effort rather than guaranteed, and the reasons are structural rather than a matter of effort. Named individually:

• Batching. Your request is served alongside other people’s. Floating-point reduction is not associative, so the same matrix multiplication summed in a different order produces results that differ in the last bits. Usually invisible; occasionally enough to flip which of two near-tied tokens wins, after which the two continuations diverge completely.
• Hardware and kernel heterogeneity. A fleet is not uniform. A different GPU generation or a different attention kernel gives numerically slightly different logits for identical input.
• The model moved. If you called mistral-small-latest, the alias may now point at a newer dated snapshot than it did last week. This is the largest source of surprise drift and the easiest to eliminate — see pinning a dated model version.
• The prompt is not identical. A timestamp in a system prompt, a re-ordered JSON key in a serialised tool schema, a trailing newline. Diff the exact bytes you sent before blaming the seed.

There is a subtler one worth adding to that list: other sampling parameters interact with the seed and are easy to leave unset. The seed fixes the sequence of random numbers, but which token each number selects depends on temperature, top_p and any penalties in force. Change the temperature and the same seed produces different text, correctly — you have re-shaped the distribution the fixed draws are applied to. So a reproducible request means pinning the whole sampling configuration explicitly, not just the seed, and not relying on defaults that are a property of the client library rather than of your code.

Divergence, when it happens, is usually total rather than partial. One different token early puts the rest of the generation on a different path, because every subsequent step conditions on it. So “mostly the same with a few words changed” and “a completely different answer” are the same failure at different positions.

The practical consequence is that a seed is a debugging aid rather than a contract. When a seeded request stops reproducing, the useful reflex is to check the list above in order — snapshot first, because it explains the largest jumps and is the only one you can actually fix — rather than to conclude the parameter is ignored.

Temperature zero is not the answer either

The instinct on reading the above is to set temperature: 0 and skip the seed. Greedy decoding does remove the sampler as a source of variance — with temperature at zero the highest-scoring token wins and the random number generator is not consulted, which is why a seed makes no observable difference at that setting.

But it does not remove the numerical variance above. Greedy decoding takes an argmax over logits that still wobble in their last bits, and when the top two tokens are close, the argmax flips. Temperature zero makes output near-deterministic and reduces the frequency of divergence considerably; it does not make it a guarantee. It also changes the output — greedy text is measurably flatter and more repetitive than sampled text — so using it as a testing device means testing something you do not ship.

Using it for something real

The seed is genuinely useful once you stop expecting a guarantee from it. Three uses that hold up:

• Labelled variation. Generate five variants of a piece of copy with seeds 1 through 5 at a normal temperature. You get diversity, and if variant 3 is the good one you can regenerate it — usually — by asking for seed 3 again.
• Bug reports. A reproduction that includes the exact request body, the dated model id and the seed is one somebody else has a chance of reproducing. Without the seed they are chasing a distribution.
• Reducing evaluation noise. Comparing two prompts on the same fixed seed removes one source of difference between the runs, so a smaller sample tells you more. It does not remove the need for a sample — a single seeded run comparing two prompts is one draw, not a result.

One thing the seed does not do, and which people expect it to: it does not make a request cheaper or faster. There is no lookup by seed. Each call runs the model from scratch and is billed for every token it produces. If what you actually wanted was to avoid paying twice for the same answer, that is a cache keyed on the request body, which you build yourself and which is exact rather than best-effort.

What the seed is not is a caching mechanism or a test oracle. If you need a test that passes on every run, assert on properties of the output — it parses as JSON, it matches the schema, it contains the required field, it is under the length budget — rather than on an exact string. That test survives a model upgrade; a golden-string assertion pinned to a seed does not, and the day it breaks tells you nothing about whether anything got worse.