Failover that happens without a retry loop in your code
Name a model, name the ones to try if it cannot be served, and set which providers you prefer. Multigrid retries transient failures on each provider and moves down the list when it runs out of attempts.
{
"model": "anthropic/claude-sonnet-5",
"models": [
"openai/gpt-5.1",
"meta/llama-3.3-70b-instruct"
],
"provider": {
"order": ["groq", "together"],
"allow_fallbacks": true,
"sort": "price"
},
"messages": [
{ "role": "user", "content": "..." }
]
}- 0
- lines of retry code
- 3
- attempts before failover
- Ordered
- fallback across providers
- Drop-in
- OpenAI-compatible request body
What it actually does
Ordered fallback chains
Send `models: ["a", "b"]` and each is tried in turn when the one before it cannot be served. Presets carry their own fallback list so callers need not repeat it.
Retries on the right errors only
408, 425, 429 and 5xx are retried on the same provider with full jitter; a 400 or a bad key is not, because it will fail identically on the next attempt.
The provider's Retry-After wins
When a provider says when it will be ready, that beats our backoff — and a Retry-After longer than the budget is treated as a signal to fail over now rather than sit and wait.
Provider preferences
`order`, `ignore` and `allow_fallbacks` pin or exclude specific providers. `sort` picks by price, throughput or latency across the routes that serve the same model.
Weighted splits
Send a percentage of traffic to a second provider — a 5% canary, or enough to keep a backup warm so failover is not the first real traffic it has seen all week.
Versioned presets
A preset is a saved model, system prompt and parameter set, callable as `@preset/name`. Every save publishes a version; `@preset/name@3` pins one, and restoring an old version is one write.
Fair questions
What happens if every model in the chain fails?
The request returns 502 and the row in Activity records how many attempts were made and which providers failed. There is no synthetic fallback response — a gateway inventing an answer it did not get would be worse than an error you can see.
Do I have to rewrite my client?
No. The routing fields are extra keys on an ordinary OpenAI-shaped body, ignored by any SDK that does not know them. The names match OpenRouter's on purpose, so a config that already works keeps working.
Can I stop it failing over?
Set `allow_fallbacks: false` and the request is pinned to the first usable route. Some workloads would rather fail than silently run somewhere else.
Works with the rest of the platform
Add your own provider keys and Multigrid routes through them — your rates, your billing relationship. We charge nothing on that traffic and it does not touch your credit balance.
Every request writes a row: model, provider, tokens, cost, latency, attempts, cache outcome and your own tags. No prompt text, no completion text — the schema has nowhere to put them.
Six rule types, evaluated inside the request path. A client that forgets to call them still gets them, and a rule can run in warn mode against real traffic before it is allowed to refuse anything.
Bring your own provider keys and pay us nothing for that traffic
Requests served on your own key are not charged and do not touch your credit balance. You still get the routing, the logging and the cost record.