JSON Mode in the Mistral API: response_format Type json_object

Setting response_format to json_object makes Mistral return valid JSON. It does not make it return your JSON, and if you omit one thing from your prompt it can make the model emit whitespace until it exhausts the context window.

The request

JSON mode is one field. Everything else about the call is unchanged:

curl https://api.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-large-2512",
    "messages": [
      {
        "role": "system",
        "content": "Extract the sender, the amount and the currency from the email. Reply with a JSON object with keys sender, amount, currency."
      },
      {
        "role": "user",
        "content": "Hi, this is Marie from Atelier Dubois. Our invoice for 1,240 EUR is attached."
      }
    ],
    "response_format": {"type": "json_object"},
    "max_tokens": 200
  }'

The content of the returned message is a string containing JSON, not a parsed object — the response envelope is the ordinary chat completion shape, and choices[0].message.content holds text you still have to run through a parser:

{
  "id": "cmpl-3d5f2a1c9b7e4f0a8c6d2e1f3a4b5c6d",
  "object": "chat.completion",
  "model": "mistral-large-2512",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"sender\": \"Atelier Dubois\", \"amount\": 1240, \"currency\": \"EUR\"}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {"prompt_tokens": 96, "completion_tokens": 28, "total_tokens": 124}
}

What is actually guaranteed

Mistral’s known limitations page states the guarantee precisely: when response_format: {"type": "json_object"} is set, the model always returns valid JSON — and, separately, that JSON mode does not guarantee adherence to a specific schema.

Read those two sentences as a pair, because together they describe exactly one property: syntactic validity. You will get balanced braces, quoted keys, no trailing commas, no prose preamble saying “Here is the JSON you asked for:”. You will not necessarily get the keys you named, the types you wanted, or the same shape twice in a row. amount can come back as the string "1,240" on one call and the number 1240 on the next, and both are valid JSON.

The mechanism is worth holding on to, because it predicts the behaviour. Constrained decoding works by masking the token distribution at each step so that only tokens which keep the output parseable are available. The grammar being enforced is “JSON”, not “JSON matching this schema”, so anything the JSON grammar permits, the model may produce.

The infinite whitespace stream

This is the failure worth knowing before you ship. Mistral’s documentation states it directly: you must include “JSON” in the system or user prompt, or the model may produce an infinite whitespace stream.

It follows straight from the masking mechanism. Whitespace is legal between JSON tokens — a parser skips it — so at any point in the output, a space or a newline is a valid continuation and is not masked out. Ordinarily the model has no reason to pick it, because the prompt has made clear that a structured object is wanted and real content scores higher. Take that instruction away and the model is being asked to produce prose while being told only prose-shaped tokens are forbidden. The distribution flattens across the tokens that remain, and the ones that remain include whitespace, indefinitely.

What you observe is a request that never returns, or a streamed response that arrives as an unbroken run of blank deltas, burning output tokens at full price until it hits the window. Two defences, and you want both:

• Say “JSON” in the prompt. Mistral’s own guidance is to explicitly ask the model to return a JSON object and describe the format, even with the parameter set. The word itself is what anchors the distribution.
• Always set max_tokens. The parameter defaults to null, which means unbounded. A JSON-mode call with no bound is the one place in this API where a prompting mistake turns into an open-ended bill rather than a bad answer. Set it to a little more than your largest legitimate object.

When json_object is the wrong tool

If you need particular keys with particular types, syntactic validity is not the property you are looking for and no amount of prompt tuning will convert it into one. Mistral offers two things that do enforce shape.

The first is a schema-typed response format: response_format also accepts json_schema, per the chat completions reference, which constrains decoding against the schema you supply rather than against bare JSON. The second is function calling — Mistral’s documentation explicitly suggests using it for structured outputs, because a tool’s parameters object is a JSON Schema and the arguments are generated against it. Forcing the call with tool_choice set to a specific tool gives you a shape-checked object and no prose branch to handle.

The reasonable default: reach for json_object when the shape is genuinely open — a summarisation whose fields vary, an extraction where missing keys are meaningful — and for a schema or a tool whenever a downstream system will index into the result by key.

One cost applies to all of these and is worth pricing in before you choose. Constraining the output shape is not free in quality terms. Masking the distribution removes tokens the model would otherwise have picked, and on a hard extraction the model sometimes wanted to say something the grammar had no room for — that a field is genuinely absent, that the input was ambiguous, that two candidate values are equally plausible. A schema with no way to express “not present” forces a value, and the value you get will be a plausible fabrication rather than an error. Design the nullable and uncertainty cases into the schema explicitly: an optional field, an enum member for “unknown”, a confidence field. Otherwise the constraint that was supposed to make your output trustworthy is the thing making it wrong.

JSON mode and streaming

You can set stream and response_format together, and the combination is more awkward than it first appears. The deltas arrive as token fragments, so what you receive is a partial JSON document that is not valid JSON at any point until the last frame. There is no moment part-way through where {"sender": "Atelier parses.

That leaves two honest options. Accumulate the whole thing and parse once at the end, in which case streaming has bought you a progress indicator and nothing else — the user cannot be shown a half-built object. Or use an incremental parser designed for partial input, which emits values as they complete and lets you render fields progressively. The second is genuinely nicer for a UI that fills in a form as the model produces it, and it is considerably more code than most people expect, because you also have to decide what a half-received string value means to your renderer.

There is a diagnostic benefit to streaming here regardless of which you pick, and it is the reason to consider it at all: the infinite whitespace failure is visible immediately. A stream of deltas whose content is nothing but spaces and newlines is unmistakable in a log, where the same event in a non-streaming call is just a request that has not come back yet. If you are debugging a JSON-mode call that hangs, re-running it with stream set to true will usually tell you in three seconds what a timeout would not tell you at all.

One thing not to do: stream a JSON-mode response straight to a browser and let client-side code attempt a parse on every frame. It will throw on almost all of them, and the exception handling will hide the one frame that mattered.

Parsing the result safely

Even with the guarantee, treat the parse as fallible. Three things can still land you with a string that does not parse:

• Truncation. If generation stops at max_tokens, you get a prefix of a valid document, which is not a valid document. Check finish_reason: length means the object is cut off and retrying with a higher bound is the fix, not repairing the string.
• Stop sequences. A stop value that happens to appear inside your JSON — a closing brace, a quote — will end generation mid-object. JSON mode and stop sequences interact badly; usually you want neither or only one.
• Empty content. A response can carry an empty content string. Handle it as a retry rather than letting the parser raise into a stack trace your caller sees.

And once it parses, validate it. The API told you it was JSON; only your schema check can tell you it was the right JSON.