Skip to content
Multigrid

Structured output

response_format reaches OpenAI-format providers untouched. Anthropic routes are not silently served — they fall back, or they refuse.

5 min read

How it works here

Multigrid does not implement schema enforcement itself. It is a passthrough: response_format is a field we do not model, so on any provider that speaks the OpenAI format — OpenAI, Groq, Together, DeepInfra — it reaches them exactly as you wrote it, and whatever guarantee that vendor offers is the guarantee you get.

Request body
{
  "model": "openai/gpt-5-mini",
  "messages": [{"role": "user", "content": "Extract the invoice total."}],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "invoice",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": { "total_cents": { "type": "integer" } },
        "required": ["total_cents"],
        "additionalProperties": false
      }
    }
  }
}

The Anthropic exception

response_format is not honoured on Anthropic routes
The Anthropic adapter builds its request field by field rather than forwarding yours, because the Messages API has a different shape. response_format is not one of the fields it builds, so Claude cannot be asked for JSON this way.

It is not ignored, though. An Anthropic route that cannot satisfy json_schema or json_object is treated as unable to serve the request: if your routing has a fallback on an OpenAI-format provider, the request goes there and succeeds. Only when every candidate is Anthropic do you get a 400 naming the limitation. You will never get prose back from a request that asked for JSON.

Two ways round it, both real today:

  • Use tool calling instead. Declare a single tool whose parameters is your schema and set tool_choice to that function. Tool schemas are translated, so this works on Claude and on everything else, from one request body. See tool calling.
  • Keep the schema and pin the route. If you want json_schema semantics specifically, restrict the request to providers that honour it with provider: { ignore: ["anthropic"] }, so a fallback cannot quietly land you on a route that ignores it.

Checking you got JSON

There is a guardrail of kind json_output that inspects the response and flags anything that does not parse. It is an output-stage rule, so the tokens have already been spent by the time it fires — it is there to tell you a model or a route has started misbehaving, not to prevent the charge.

Something here disagrees with what the API actually did? That is a bug in this page, and worth reporting.

Report it