The auto router

The rule

multigrid/auto picks the cheapest chat model that can actually do the job. That is the whole rule. The choice and its reasoning come back on the response as model and auto_reason, so it is never a black box.

{
  "model": "multigrid/auto",
  "messages": [{"role": "user", "content": "Label this ticket: billing, bug or feature."}]
}

// → "multigrid": {
//      "model": "meta/llama-3.1-8b-instruct",
//      "requested_model": "multigrid/auto",
//      "auto_reason": "cheapest model that fits 24 prompt tokens",
//      …
//    }

“Cheapest” is measured on what this request would cost on each candidate — prompt tokens at the prompt rate plus your requested output at the completion rate — not on a headline price. A model with a cheap prompt rate and an expensive completion rate is the wrong answer for a long reply, and sorting on either number alone would pick it.

What makes a model eligible

• It fits. Prompt tokens are estimated at roughly four characters each, given 30% headroom, and added to the output you asked for. A model whose context lengthContext lengthHow much the model can read at once, counting your whole conversation and any documents you attach. Go over it and the request is refused rather than truncated.A 128,000-token context is roughly a 300-page book. Bigger is not automatically better: you pay for every token you put in it. is smaller than that is dropped — a context overflow is a hard failure, not a degraded answer.
• It has the capabilities the request needs. Tools if you sent tools; vision if any message carries an image.
• Something can serve it. A model with no provider we hold a usable key for is skipped, so auto never resolves to something that then cannot run.

What it does not do

It also only ever picks chat models. Embedding, image and rerank models have no meaningful completion price, so a cheapest-first list would put them at the front and then fail upstream on every request.