Rate Limit Differences You Hit When Switching Providers

A migration that passes every functional test in staging gets throttled in its first hour of production traffic, and the reason is almost never that the new limit is lower. It is that the new limit is measured in a different unit, over a different window, against a different scope.

The four axes

“Their rate limit is 10,000” is not a comparable statement until you have pinned all four of these. Every provider picks a point in this space and they do not pick the same point.

• Unit. Requests, or tokens, or both enforced simultaneously — and if tokens, whether input and output share one budget or have separate ones.
• Window. Per minute is the most common, but per day and per month exist as second-order caps, and some deployments enforce a concurrency limit instead of a rate: N requests in flight at once, with no time term at all.
• Scope. Per API key, per project, per organisation, per model, per model-and-region, or per deployment.
• Tier. Whether the number you get is fixed, assigned by a usage tier that rises with spend history, allocated from a quota you distribute yourself, or drawn from a shared pool.

Unit: requests, input tokens, output tokens

The most consequential difference between provider shapes is whether output tokens are metered separately from input tokens.

OpenAI’s documented model is requests per minute and tokens per minute enforced together, with requests per day as an additional cap on some models, and separate queue limits for the Batch API. Anthropic’s documented model splits the token budget: requests per minute, input tokens per minute, and output tokens per minute are three separate counters, any of which can be the one you exhaust.

That difference has a real consequence for a workload with an unusual shape. A summarisation service sends very long inputs and receives short outputs. Under a combined token budget, the input dominates and you hit the ceiling on input volume. Under split budgets, the same workload may have enormous output headroom it can never use, while the input counter is the binding constraint — so raising overall throughput requires an input-limit increase specifically, and a request phrased as “we need more tokens per minute” will not get you there.

The reverse shape — short prompts, long generations, such as a drafting tool — is the one that breaks unexpectedly when moving to split budgets, because output tokens per minute is typically the smallest of the three numbers and nobody was watching it before.

Some enterprise deployments do something different again: you buy or allocate a throughput quota per deployment and the provider derives the request rate from it by a documented ratio, so raising your token allocation raises your permitted request rate as a side effect. If you are moving onto that shape, the unit you provision is not the unit you were previously rate-limited in, and the conversion is the thing to look up first.

Scope: what the counter is attached to

Scope is where migrations break silently, because staging usually has one key and production has many.

If limits are organisation-wide, every service in your company shares one counter, and the batch job that runs at 02:00 is competing with the interactive path. If they are per-key or per-project, you can isolate workloads by issuing separate keys — which is a design decision you get to make once, at migration time, and which is painful to retrofit because it means re-issuing credentials across services.

If limits are per model, then a fallback chain that redirects traffic from a saturated model to a second model gets a fresh budget, and routing between models is a throughput mechanism as well as an availability one. If limits are organisation-wide across models, the same fallback buys you nothing at all and you will spend a day confused about why.

Regional scope is the third trap. Where limits attach to a model-and-region pair, capacity in one region does not help a saturated one, and a migration that consolidated onto a single region for latency reasons has quietly consolidated its rate limit too.

Tier: the limit you get on day one is not the limit you had

This is the specific thing that catches migrations, and it is structural rather than accidental. Both major providers assign limits by a usage tier that rises with cumulative spend and account age. Your incumbent account has years of history and sits near the top. Your new account has none and starts at the bottom.

So the comparison that matters is not “their published maximum versus our current usage”. It is “the tier we will actually be in during week one versus our current peak-minute usage”. Those can differ by more than an order of magnitude, and the gap closes on the provider’s schedule, not yours.

Three things follow, and all three are cheaper to do before the cutover than during it:

1. Measure your peak minute, not your average. A monthly token total divided by minutes in a month understates a bursty workload by a large factor, and rate limits are enforced against the peak.
2. Find out from the new provider’s documented tier table which tier your expected spend places you in, and compare that tier’s limits against the peak. Do this before committing to a date.
3. Start spending early. Where a tier is unlocked by cumulative spend and elapsed time, moving a small amount of non-critical traffic across weeks before the real cutover advances the account through the tiers while nothing depends on it.

Reading the limit off the response

Do not infer your limits from documentation. Read them off a real response, because the headers report what is being enforced for your account right now.

OpenAI returns x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests and the corresponding -tokens triple. Anthropic returns anthropic-ratelimit-requests-limit, -remaining and -reset, plus separate anthropic-ratelimit-input-tokens-* and anthropic-ratelimit-output-tokens-* families reflecting the split budgets described above. Both return retry-after on a 429, and honouring it is strictly better than a fixed backoff because it is the server telling you when the window rolls.

Log the remaining counters as a gauge from day one of the migration, not after the first incident. The useful alert is not on 429 responses — by then you are already dropping traffic — but on the remaining fraction dipping below a threshold, which gives you warning while there is still headroom. And distinguish 429 for rate limiting from 429 for exhausted credit or quota: they share a status code, they need opposite responses, and the difference is in the error body’s type field rather than in the status. Retrying a quota exhaustion forever is how a migration turns into an outage.