How a request is routed

The order

Everything a chat request touches happens in one fixed sequence. It is worth knowing because it explains most surprises — particularly which errors cost money and which do not.

• Preset expansion. A @preset/… model id is resolved first, because the preset decides the model and the model decides the price, the routes and the cache key.
• Validation of our own fields. metadata, attachments and plugins are checked here — the last point at which refusing costs nothing.
• Rate limit. Your key’s per-minute bucket. Before the idempotency claim, so a throttled request never takes ownership of a key it will not fill in.
• Cache read. Only if the request opted in. A hit returns here, having cost nothing and reached no provider.
• Idempotency claim. Taken before anything can reach a provider, which is the property that makes it work.
• Route resolution. The model chain becomes a list of provider routes, filtered and ordered by your preferences.
• Guardrails, input stage. Your own rules. A block here is free.
• Balance pre-flight. Priced against the worst legal outcome of the request. After this point tokens can be spent.
• The walk. Each candidate is retried a few times before the list moves on. First success wins.
• Record and charge. One requests row, one ledger entry, exactly once.

Building the candidate list

The chain is your model plus anything in models, de-duplicated. For each model in turn we resolve every enabled provider route, apply your provider preferences, and keep the ones we can actually serve — meaning we have an adapter for the provider and a credential that can pay for it. Your own provider key is preferred over ours wherever both exist.

Routes for the wrong endpoint are dropped, and if that leaves nothing the error names the right one: an embedding model sent to /chat/completions gets a 400 saying to use /embeddings, rather than an upstream error nobody can trace back.

Why a route gets dropped

A dropped route is not an error. It is only an error when every route is gone — and then the message says which of these it was.

Seeing it before you send

The routes for any model, in the order they will be tried, with the price on each and whether we currently hold a key for it:

curl https://api.multigrid.ai/v1/models/meta/llama-3.3-70b-instruct/endpoints

Your own account’s view of the same thing — including which providers your BYOK keys cover — is on the Routing page, which also lists the requests that actually needed more than one attempt.

There is no policy object

Routing is expressed on the request, not stored as a named, versioned policy resource. If you want a configuration you can change without a deploy, save it as a preset — that is versioned, pinnable and callable by name, and it carries the provider preferences along with the model and the prompt.