Fixing a 429 Rate Limit on Azure OpenAI

A 429 from Azure OpenAI has four distinct causes, and three of the four are made worse by the obvious fix. Start by reading which one you have.

The error you pasted

The rate-limit variant of the response looks like this, with the operation name, API version and retry interval varying:

HTTP/1.1 429 Too Many Requests
retry-after-ms: 6000
x-ratelimit-limit-tokens: 60000
x-ratelimit-remaining-tokens: 0

{
  "error": {
    "code": "429",
    "message": "Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-10-21 have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 6 seconds."
  }
}

That reference to an “S0 pricing tier” is the sentence that sends people down the wrong path. S0 is the resource SKU and there is no higher one to buy; the limit being hit belongs to the deployment, and it is set by the capacity you assigned when you created it. Nothing about the resource tier changes it.

Four causes wearing one status code

Microsoft’s quota documentation enumerates them, and the distinguishing evidence is in the message and the headers rather than in the status:

• Rate limit exceeded. Message contains “have been limited” or “Rate limit is exceeded”. Your traffic exceeded the deployment’s TPM or RPM. This is the one that quota fixes.
• System capacity throttling. Message says the service is temporarily unable to process your request, or that the system is experiencing high demand. Backend capacity, not your allocation. More quota changes nothing.
• Temporary rate limit adjustment. No message change at all — the tell is that x-ratelimit-limit-tokens in the response is lower than the TPM you configured. Microsoft documents that standard deployments share a resource pool and that the service may temporarily reduce a deployment’s effective limit to protect reliability, typically resolving within a few hours.
• Token budget consumed by parameters. The limit triggers while your usage metrics look low. This is the max_tokens case below.

The third is the one worth internalising, because no amount of investigation on your side will explain it and it is invisible unless you compare the header against your configuration. Microsoft states plainly that many customers misread capacity-related 429s as quota problems and request increases that cannot help.

Why it fires below your usage metrics

Rate limiting and billing count different things at different times. Microsoft documents that the TPM limit is evaluated against an estimated maximum processed-token count computed when the request arrives, from the prompt text, the max_tokens setting and the best_of setting. Billing counts actual tokens after the fact.

So a request with a 4,000-token max_tokens that returns 150 tokens consumes 4,000-ish of your rate budget and 150 of your bill. Azure Monitor shows the 150. The limiter saw the 4,000. That single mismatch accounts for most “we are at 20% of quota and getting throttled” reports, and it is the cheapest thing to fix because it costs nothing.

Two smaller contributors: Microsoft notes the estimate is based partly on character count and is deliberately approximate, so a limit can trigger slightly earlier than an exact token count would suggest; and rejected requests still count toward rate limiting even when they never appear in billed-token metrics.

RPM has its own version of this. Microsoft documents that the limiter expects requests spread evenly and evaluates over windows of typically 1 or 10 seconds — its worked example is that a 600 RPM deployment throttles if more than 10 requests arrive in any 1-second window. A job that fires 300 requests at the top of the minute and idles for the rest is over the limit in the only window that is measured.

Read the headers, not the dashboard

Azure OpenAI returns rate-limit state on every response, not just on 429s:

x-ratelimit-limit-requests       60
x-ratelimit-limit-tokens         150000
x-ratelimit-remaining-requests   59
x-ratelimit-remaining-tokens     149984
x-ratelimit-reset-requests       10
x-ratelimit-reset-tokens         300
retry-after-ms                   2000    (429 responses only)

Log x-ratelimit-limit-tokens on every successful call. It costs nothing and it is the only way to distinguish cause three from cause one: if that number is below your configured TPM, the service has temporarily reduced your effective limit and the correct action is to back off, not to file a request.

A 429 from a provisioned deployment is a fifth thing again, and none of the above applies to it. Microsoft documents that rate limits on provisioned throughput are calculated differently — by utilisation rather than by a TPM counter — so there is no quota to raise and no shared pool to be protected from. A provisioned 429 means the capacity you bought is full, and the remedies are more PTUs, a smaller request shape, or spillover to a standard deployment in the same resource. That arithmetic is worked through on the provisioned throughput page.

Fixing each cause

1. Set max_tokens to what you actually need. Microsoft’s own first recommendation. If responses are around 200 tokens, do not send 4,000. Set best_of to 1 unless you need multiple completions — each increment multiplies the count against your limit.
2. Honour retry-after-ms, then back off with jitter. The OpenAI Python SDK from v1.0 retries 429s automatically with exponential backoff, defaulting to two attempts; max_retries raises it. If you wrap it in your own retry library, set max_retries=0 on the client or every one of your attempts triggers up to two more underneath and you multiply the load you are trying to reduce.
3. Smooth the arrival pattern. A queue with a paced consumer fixes the burst case that no quota increase will. Ramp new workloads gradually rather than switching them on at full volume.
4. Check the deployment, not the subscription. Quota approved at subscription level does nothing until it is assigned to the deployment taking the traffic. Rebalancing capacity from an idle deployment is instant and free; see the quota page.
5. If it is sustained and you are below approved quota, escalate. Microsoft’s own guidance is that occasional transient 429s on standard deployments are expected behaviour, and that sustained production 429s below approved quota warrant a support request rather than another increase.

The structural fix, when retries stop being enough, is a second place to send the request — another region, another deployment type, or provisioned capacity that does not share a pool.