JSON Mode in the Cohere API: response_format and Schema Support

response_format makes Cohere return valid JSON by constraining what the model is allowed to emit, not by asking it nicely. Supply a schema and the constraint extends to the field names and types. Neither makes the contents true.

The response_format field

The minimal form asks for a JSON object and nothing more about its shape:

curl https://api.cohere.com/v1/chat \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "command-r-plus-08-2024",
    "message": "Extract the order id, the carrier and the delay in days: order A-1204 went out via PostNL and arrived five days late.",
    "response_format": {"type": "json_object"}
  }'

The response text is then a JSON document as a string — you still parse it yourself; the API does not hand you an object. What has changed is that the parse will not fail on a preamble like “Sure! Here is the JSON:”, on a Markdown code fence, or on a trailing explanation, because the model was never able to produce those characters.

Cohere documents JSON output as supported on the Command R generation from the August 2024 snapshots onward and on newer models. On an older model the parameter is not silently ignored — it is rejected — which is the better of the two behaviours and worth knowing when a request that works on one pinned snapshot fails on another.

Adding a schema

A bare json_object guarantees syntax and nothing else: field names, nesting and types are all still the model’s choice, and they will drift between calls. Supplying a schema fixes them.

{
  "model": "command-r-plus-08-2024",
  "message": "Extract the order id, the carrier and the delay in days: order A-1204 went out via PostNL and arrived five days late.",
  "response_format": {
    "type": "json_object",
    "schema": {
      "type": "object",
      "required": ["order_id", "carrier", "days_late"],
      "properties": {
        "order_id":  {"type": "string"},
        "carrier":   {"type": "string", "enum": ["DHL", "PostNL", "DPD"]},
        "days_late": {"type": "integer"}
      }
    }
  }
}

Which produces exactly this shape, every time:

{
  "text": "{\"order_id\": \"A-1204\", \"carrier\": \"PostNL\", \"days_late\": 5}",
  "finish_reason": "COMPLETE"
}

The enum is the most valuable line in that schema. It is the difference between a carrier field that might contain "PostNL", "postnl" or "Post NL" depending on the day, and one that is a member of a set your database already knows about. Note also that days_late came back as the integer 5 rather than the string “five” that appeared in the input: the type constraint is applied during decoding, so the model cannot emit a quote where a digit is required.

In /v2/chat the equivalent uses {"type": "json_object", "json_schema": {...}}. The key name differs between versions; the schema inside it does not.

Which schema keywords survive

Constrained decoding is implemented by restricting the tokens the model may emit at each position, and that only works for constraints expressible as “what is legal next”. The keywords that reliably work are the structural ones: type, properties, required, items, enum. Cohere documents the supported subset in its structured outputs documentation, and it is a subset — not the full specification.

The keywords to be suspicious of are the ones that constrain a value after it exists: minimum and maximum on numbers, minLength, pattern, minItems. A decoder cannot always know at token three that the number it is building will end up below your minimum. Depending on the model and the version these are either rejected outright or accepted and not enforced, and the second is the dangerous one because it looks like it worked.

The reliable posture is to express in the schema only what the schema can enforce, and to validate the parsed object against your full schema afterwards regardless. The schema in the request is a generation aid; the validator in your code is the guarantee.

What constrained decoding does and does not fix

This is the distinction that decides how much you can trust the result. Constrained decoding masks the token distribution so that only tokens consistent with the grammar have non-zero probability. It is a mechanical guarantee about form.

• Fixed: malformed JSON, prose around the JSON, code fences, missing required fields, wrong types, values outside an enum.
• Not fixed: a field populated with a plausible invention because the source text did not contain it. A required field the model cannot answer will be filled with something schema-valid rather than left out — that is what required means to a decoder. If a field may genuinely be unknown, its type must permit null, or you are asking the model to guess and formally forbidding it from declining.
• Not fixed: truncation. A response that hits the output ceiling mid-object comes back as invalid JSON with a finish_reason of MAX_TOKENS. The constraint governs which tokens are legal, not how many there are, so parsing can still fail — check the finish reason before you blame the parser.

A schema that survives real data

Most schemas are written against the example that prompted them and then meet input that does not fit. Four adjustments make the difference, and all of them follow from the fact that the decoder will always produce something schema-valid, whatever the input said.

{
  "type": "object",
  "required": ["order_id", "carrier", "days_late", "confidence"],
  "properties": {
    "order_id":  {"type": "string"},
    "carrier":   {"type": ["string", "null"], "enum": ["DHL", "PostNL", "DPD", null]},
    "days_late": {"type": ["integer", "null"]},
    "confidence": {"type": "string", "enum": ["high", "low"]},
    "notes":     {"type": "string"}
  }
}

• Make absent values expressible. A union with null gives the model a way to say the text did not contain a carrier. Without it, the only schema-valid output is a guess, and you have formally required the model to invent one.
• Keep the enum closed but complete. An enum is the strongest constraint available and the most useful, but an enum missing a real category forces every instance of it into the nearest wrong value. Include a null or an explicit "other" member where the world is open.
• Ask for a confidence field. Not because the model is calibrated — it is not — but because a two-value flag gives you a cheap routing signal for human review, and a model that had to emit "low" is measurably more likely to have been guessing than one that emitted "high".
• Order the properties the way you want them reasoned. Generation is sequential, so a field emitted early is in the context when later fields are produced. Putting an evidence or quote field before the value it justifies is the structured-output equivalent of asking for the reasoning first, and it costs only the tokens of the field.

The complementary discipline is on the reading side: parse, then validate against the same schema in your own code, then handle nulls explicitly. Treating a null as a failure of the model is usually wrong — it is frequently the correct answer, and the pipeline that crashes on it is the one that had no way to express “not present” in the first place.

When a tool call is the better shape

There is a second way to get structured output from Command: define a tool whose parameters are the structure you want, and read the arguments off the call instead of executing anything. The arguments arrive already structured in the tool call shape, with no string to parse.

There is a cost difference too, and it runs the other way from what people assume. A tool definition is sent on every request in the conversation; a response_format schema is also sent every time, but only one of them additionally invites the model to spend tokens on a plan before it answers. For a single-shape extraction task running at volume, response_format is the leaner request as well as the simpler one.

The choice between them is not about reliability so much as about how many shapes you need. response_format gives one shape per request and is the right answer for extraction, where you always want the same object. Tools let the model choose between several — an extraction, a clarifying question, a refusal — each with its own schema, whichresponse_format cannot express because it admits exactly one.