Model Cascade Savings Calculator

Prices cheap-first routing against sending everything to the expensive model, and gives the escalation rate at which the saving disappears.

Requests per month

Escalation rateShare of requests the cheap model hands upward

Routing overheadA classifier or judge call, in dollars per 1,000 requests. 0 if the cheap model decides for itself.

$/1K req

Cheap model — input tokens per request

Cheap model — output tokens per request

Cheap model — input rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Cheap model — output rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Expensive model — input tokens per requestUsually the same prompt, sent again

Expensive model — output tokens per request

Expensive model — input rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Expensive model — output rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Saved per month by going cheap first

$1,704

The cascade stops paying at an escalation rate of 96.8%. You entered 20.0%, so you have 76.8% of headroom.

Cheap call, including routing overhead: $0.00036
Expensive call: $0.01
Cascade, blended at 20.0% escalation: $0.0026
Baseline — everything to the expensive model: $0.01
Difference per request: $0.0085
Baseline bill per month: $2,220
Cascade bill per month: $516.00
Escalations per month: 40,000
Break-even escalation rate: 96.8%
Escalation headroom: +76.8%

What this assumes: an escalated request pays for BOTH calls in full: the cheap attempt is already billed when the escalation happens, and the prompt is re-sent to the expensive model, so the input tokens are paid twice. That is what makes the break-even rate lower than people expect. The cheap and expensive models are assumed to see the same input; if your escalation adds the first answer to the second prompt, raise the expensive input tokens to match. No cache is modelled, and no quality difference is — the cost of escalating too rarely is a worse answer, and this tool cannot price that.

The break-even escalation rate is the number to remember

A cascade sends everything to a cheap model first and promotes the hard cases. The saving is not the price gap between the two models, because every promoted request pays both bills. Write C for the cheap call, E for the expensive one and p for the share that escalates: the cascade costs C + pE per request against a baseline of E, so it wins only while p is below 1 − C/E. That is the break-even rate, and it is the whole design constraint in one expression.

Two things follow that are not obvious. First, the break-even is set by the RATIO of the two models, not the gap. If the cheap model is a twentieth of the price, you can escalate 95% of traffic and still be ahead; if it is half the price, you cannot escalate more than half. A cascade between two similarly-priced models has almost no room and is rarely worth the complexity. Second, the escalation rate you measure in testing is not the one you will run at. Traffic drifts, prompts get longer, and a router tuned to escalate 15% of last quarter's questions will escalate more of this quarter's. Leave headroom, and alert on the rate rather than only on the bill.

What this leaves out is quality, which is the real reason cascades fail. Requests that should have escalated and did not do not appear as cost; they appear as bad answers. Measure the escalation rate together with whatever quality signal you have, and treat a suspiciously low rate as a routing bug rather than a saving.

Model Cascade Savings Calculator

The break-even escalation rate is the number to remember

The reasoning behind this

Related tools