The cheapest token is the one you never send
An exact-match response cache, off unless a request asks for it. Identical calls come back from our database instead of the provider, and a hit is charged nothing.
- Opt-in
- nothing is cached unless you ask
- $0
- charged for a cache hit
- 24h
- maximum TTL
- Per-account
- keys, never shared
What it actually does
Exact match only
Set `X-Multigrid-Cache-Ttl` or `cache: { ttl }` on a request and a byte-identical repeat is served from the cache. There is no semantic or near-duplicate matching — see the FAQ.
Scoped correctly
The account id is inside the cache key hash, not beside it. So are the model, messages, temperature, top_p, max_tokens, stop, tools, response_format and seed — change any of them and it is a miss, not a stale answer.
You set the lifetime
One hour by default, one day at most. Past that, "the model changed" is the likelier explanation than "the answer is still right".
Visible hit rate
The cache page shows hit rate over 30 days, what those hits would have cost upstream, and the most-hit entries. Keys are shown as hashes; the prompt is not recoverable from them.
Works with streaming
Only complete non-streamed responses are written, but a streaming caller still gets a hit — it is replayed as SSE so the SDK on the other end never has to care.
Provider caching is counted
When a provider does its own prompt caching, the cached portion of the prompt comes back in the usage numbers and is priced at that provider's cached rate rather than the full input rate.
Fair questions
Is there semantic caching?
No. Only exact matches are served. Serving a near-duplicate means deciding that two different prompts deserve the same answer, and getting that wrong is invisible to the caller — so it is not something to ship with a default threshold and hope.
Why is caching off by default?
Because a gateway that cached automatically would make every customer's application quietly non-deterministic — the same call returning a frozen answer an hour later, with no way to tell from the outside. It has to be asked for, per request.
Does caching break personalisation?
Not if the personalised part is in the messages, which it almost always is. The full message array is part of the key, so two users never share an entry unless their whole request matches.
Works with the rest of the platform
Name a model, name the ones to try if it cannot be served, and set which providers you prefer. Multigrid retries transient failures on each provider and moves down the list when it runs out of attempts.
Add your own provider keys and Multigrid routes through them — your rates, your billing relationship. We charge nothing on that traffic and it does not touch your credit balance.
Every request writes a row: model, provider, tokens, cost, latency, attempts, cache outcome and your own tags. No prompt text, no completion text — the schema has nowhere to put them.
Bring your own provider keys and pay us nothing for that traffic
Requests served on your own key are not charged and do not touch your credit balance. You still get the routing, the logging and the cost record.