Retry and Failover Cost Calculator

Expected attempts under your retry policy, what the failures cost, and how much you spend on requests that never produce an answer.

Requests per monthRequests you intend to make, not attempts

Input tokens per attempt

Output tokens per attempt

Input rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Output rateUS dollars per 1,000,000 tokens. Example figure — replace it with yours.

$/1M

Failure rate per attemptTimeouts, 5xx, refusals, invalid output — anything you retry on

Retries after the first attempt0 means one attempt and no more

Are failed attempts billed?The question that decides whether retries are cheap or ruinous

Tokens produced before failingShare of a full attempt. Only used for 'partly'.

Fail over to a second provider?After the retries on the primary are exhausted

Fallback attempt costMultiple of a primary attempt. 1 = same price.

Fallback failure ratePer attempt on the fallback

Monthly bill including retries and failover

$2,021

5.2% more than the $1,920 you would pay if nothing ever failed. 3 requests a month end with no answer, and they cost $0.08.

Cost of one attempt: $0.0096
Attempts allowed on the primary: 3
Expected attempts per request: 1.086
Expected failed attempts per request: 0.087
Billed fraction of a failed attempt: 60%
Expected cost per request: $0.01
Retry overhead on the bill: 5.2%
Chance all primary attempts fail: 0.05%
Failover spend per month: $1.46
Requests ending with no answer: 3 / month
Spent on requests that never answered: $0.08
Bill if nothing ever failed: $1,920

What this assumes: attempts fail independently at the same rate, which is the friendly case — real failures arrive in bursts, and a provider incident makes every retry in the burst fail together, so treat this as a floor. A failed attempt that produced tokens is still billed: that is the field in the middle, and it is the one people get wrong. Timeouts are the worst case, because the model generated a full answer that you then threw away; a 429 before any work usually costs nothing. Backoff delay is not modelled — this is a bill, not a latency budget. A month is 30 days.

The attempts you pay for and the answers you get

A retry policy is a multiplier on your bill and almost nobody writes it down. With a per-attempt failure rate f and a cap of m attempts, the expected number of attempts per request is (1 − f^m) / (1 − f). At a 3% failure rate that is barely above 1 and the overhead is a rounding error. At 30% it is meaningfully more than 1, and at 60% with three retries you are paying nearly twice per successful answer.

The part most people miss is the billing question in the middle of the form. A failed attempt that produced tokens is still billed. A request that streamed 400 tokens and then timed out at your client generated those tokens on someone's GPU, and disconnecting does not un-bill them. A response that came back complete but failed your JSON validation is billed in full — the model did its work, your parser rejected it. The cheap failures are the ones that fail before generation: rate limits, auth errors, a refused connection. So a policy that retries on validation errors is expensive in a way that a policy retrying on 429s is not, and the two are usually written in the same line of code.

Then there is the line that ends the discussion: money spent on requests that never produced an answer. Every attempt failed, every attempt was billed, and the user saw an error. That number is the one to take to a conversation about whether the retry cap should be three or one, and whether the fallback provider is worth wiring up.

Retry and Failover Cost Calculator

The attempts you pay for and the answers you get

The reasoning behind this

Related tools