JSON Mode and Structured Output in Grok

response_format takes three values on the xAI API and only one of them constrains the shape of the output. The difference between json_object and json_schema is the difference between valid JSON and the JSON you asked for.

Three values, two different mechanisms

xAI’s structured outputs guide documents response_format.type as taking:

• “text” — the default. Prose, whatever the model decides to produce.
• “json_object” — output constrained to be syntactically valid JSON, of no particular shape. Keys, nesting and types are whatever the model chose.
• “json_schema” — output constrained to match a schema you supply in response_format.json_schema.

The gap between the second and third is where integrations break. Valid JSON with the key spelled invoiceNumber instead of invoice_number parses cleanly and then fails at the field access, which is a worse failure than a parse error because it arrives further from its cause. If you have a downstream consumer with a contract, json_object is not the mode you want. It is for the case where you genuinely only need well-formed JSON — a scratch extraction, a prototype — and are willing to read whatever comes back.

Building the schema request

The schema is ordinary JSON Schema. In raw HTTP:

{
  "model": "grok-4.5",
  "messages": [
    { "role": "system", "content": "Extract invoice data into JSON format." },
    { "role": "user", "content": "Vendor: Acme Corp, 123 Main St, Springfield 62704 IL. Invoice INV-2025-001 dated 2025-02-10. 5 x Widget A at 10.00, 2 x Widget B at 15.00. Total 80.00 USD." }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "invoice",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "vendor_name":    { "type": "string" },
          "invoice_number": { "type": "string" },
          "invoice_date":   { "type": "string" },
          "line_items": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "description": { "type": "string" },
                "quantity":    { "type": "integer" },
                "unit_price":  { "type": "number" }
              },
              "required": ["description", "quantity", "unit_price"],
              "additionalProperties": false
            }
          },
          "total_amount": { "type": "number" },
          "currency":     { "type": "string" }
        },
        "required": ["vendor_name", "invoice_number", "invoice_date",
                     "line_items", "total_amount", "currency"],
        "additionalProperties": false
      }
    }
  }
}

The constrained response is the object and nothing else — no preamble, no code fence, no “Here is the extracted data:”:

{
  "vendor_name": "Acme Corp",
  "invoice_number": "INV-2025-001",
  "invoice_date": "2025-02-10",
  "line_items": [
    { "description": "Widget A", "quantity": 5, "unit_price": 10.0 },
    { "description": "Widget B", "quantity": 2, "unit_price": 15.0 }
  ],
  "total_amount": 80.0,
  "currency": "USD"
}

Because xAI’s endpoint is OpenAI-compatible, the SDK helper works too: xAI documents client.beta.chat.completions.parse(...) with a Pydantic model passed as response_format, which derives the schema from the class. That is the ergonomic route in Python, and it is the same route as the OpenAI structured outputs API, which is the point of the compatibility.

Why the guarantee is mechanical

It is worth knowing why a schema can be guaranteed when a prompt instruction cannot, because the reason determines what else follows from it.

A model emits a probability distribution over its whole vocabulary at every step, and a sampler picks from it. Constrained decoding inserts itself between those two: the schema is compiled into a grammar, the grammar tracks where in the output the model currently is, and at each step every token that could not legally come next has its probability forced to zero before the sampler runs. Part-way through a property name, the only continuations that survive are ones completing a name you declared. The model is not being persuaded to comply. It is being prevented from doing anything else.

Three consequences fall straight out of that, and all three are otherwise discovered by accident.

• Prompting still matters, for content. The grammar decides which tokens are legal; the model still chooses among the legal ones. A schema will not tell it that dates are ISO 8601 or that currency is a three-letter code. Put that in the property description fields — they are read as prompt text — or in the system message.
• An unsupported feature cannot be silently approximated. Either the compiler can express your construct as a grammar or it cannot. That is why xAI qualifies the guarantee with “supported schema features”, and why the right response to a rejected schema is to simplify it rather than to retry it.
• Fields are produced in schema order, not importance order. The model commits to earlier properties before it has generated the later ones. If a field benefits from the model having worked something out first — a classification that depends on evidence, say — declare the evidence property before the conclusion. Ordering a schema is a real lever on output quality and costs nothing.

The strict flag and additionalProperties: false are worth setting together and for different reasons. strict is what opts the request into the guaranteed path at all rather than a best-effort one; additionalProperties: false is what stops the model inventing a notes key it thought would be helpful. Without the second, a schema describes a minimum rather than a shape, and your parser meets fields nobody planned for.

What the guarantee covers

xAI’s documentation states that when you use supported schema features, the response is guaranteed to match your schema. That is a strong claim and it is a claim about mechanism, not about effort: a constrained decode masks tokens that would violate the grammar, so the model is unable to emit a key you did not declare or a string where you asked for a number.

The qualifier matters as much as the guarantee. “Supported schema features” is not all of JSON Schema. Exotic constructs — intricate conditional subschemas, unusual format assertions, recursive references — are where implementations differ, and a schema that validates in your test suite is not automatically one the decoder supports. Keep schemas boring: objects, named properties, arrays of objects, enums, required, additionalProperties: false.

What it does not cover

• Truth. A schema constrains shape, not content. A required invoice_number will be filled in even when the document has none, because the grammar demands a string. If a field may genuinely be absent, model that in the schema — a nullable type, or leaving it out of required — rather than forcing a fabrication.
• Completeness. If generation stops early the output is truncated JSON, which is invalid JSON. Check finish_reason for length before you parse — the mode makes valid output likely, and a token ceiling overrides it.
• Streaming usefulness. A stream of a constrained object is a stream of an incomplete object. There is nothing to parse until the last chunk, so structured output and progressive rendering do not combine — accumulate, then parse.
• Refusals. If the model declines, you get a refusal in ordinary prose rather than a typed field, and it will not satisfy your schema. Handle the case where a schema-constrained call comes back unparseable without assuming a bug.
• Cost. A schema is prompt tokens on every request, and a large nested one is not small. It is charged at the input rate like everything else.

When a forced tool call is better

There are two ways to get structure out of Grok, and they suit different jobs.

Use response_format when the output is the document — extraction, classification, a record you are going to store. The response body is the object and there is nothing to unwrap.

Use a forced tool call — tool_choice naming one function — when the structure is an instruction: an action to perform, with arguments. You get the same schema-shaped guarantee on the arguments, you keep the model’s ability to answer in prose when no action is warranted, and you are already inside the loop you need for executing it. The request shape for that is in the function calling page, and the same distinction appears at other providers — Anthropic’s JSON-via-tool-choice pattern is the same idea with different field names.

One caution when routing across providers: response_format is not portable in either its spelling or its guarantee. Gemini expresses the same intent through a response schema on the generation config, and the set of supported schema features differs everywhere. A schema that decodes cleanly on Grok is a schema to re-test, not to assume.