Caching

Opt-in, always

Nothing is cached unless the request asks for it. A gateway that cached by default would silently make every application non-deterministic — the same call returning a frozen answer an hour later, with no way to tell from the outside that it had.

Ask for it with a header carrying the number of seconds you want the answer kept:

curl https://api.multigrid.ai/v1/chat/completions \
  -H "Authorization: Bearer $MULTIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Multigrid-Cache-Ttl: 3600" \
  -d '{
    "model": "openai/gpt-5-nano",
    "messages": [{"role": "system", "content": "Classify as billing, bug or feature."},
                 {"role": "user", "content": "My card was charged twice."}]
  }'

# A hit answers with:
#   X-Multigrid-Cache: hit
#   X-Multigrid-Cost-Usd: 0.000000
#   "multigrid": { "cache": "hit", "cost_micros": 0, "saved_usd": 0.000021, … }

A hit costs nothing, reaches no provider, and reports what it saved. A streaming caller gets a hit replayed as SSEStreamingThe answer arrives word by word as it is written, instead of all at once at the end. It costs the same; it just feels far faster., because their SDK is already in streaming mode and would choke on anything else — in one chunk, since pretending to type it out slowly would be theatre.

What makes two requests the same

The cache key is a SHA-256 over twelve named fields: your account id, model, models, the whole message list, temperature, top_p, top_k, max_tokens, stop, tools, tool_choice, response_format and seed.

stream is deliberately not part of the key: the same question streamed and unstreamed has the same answer, so a streaming caller can be served from a non-streaming caller’s entry. Your user field and routing hints are excluded too — they change who asked and where it ran, not what comes back.

Matching is exact. Two prompts differing by one character are two different requests.

Rules and limits

Hit counts and the last hit time are kept per entry, and the Cache page shows what is live and what it has saved you.

Provider-side prompt caching

This is a different thing that you also benefit from, and it is not something you switch on here. Several providers cache the leading portion of a prompt on their own side and charge a fraction of the normal rate for it. When they report that, the count arrives as usage.prompt_tokens_details.cached_tokens and those tokens are billed at the route’s cached rate rather than the full prompt rate.

So: the cache above skips the provider entirely and costs nothing; prompt caching still calls the provider and costs less. Both show up on the same response, and only the first is yours to control.