Migrating Retry-After Header Handling Between Providers

Every provider rate-limits, and every provider tells you when to come back. They do not tell you in the same header, the same units, or with the same meaning, and a backoff policy written against one of them degrades quietly against another.

What the header actually says

Retry-After is not a vendor invention. It is defined by the IETF in RFC 9110, section 10.2.3 (published June 2022), which says a server may send it with a 503 to indicate how long the service is expected to be unavailable, and with a 3xx redirect, and that it is defined for 429 by RFC 6585. Its value is either a non-negative number of seconds or an HTTP-date.

Both forms are legal and a conforming client must handle both. That is the first thing a migration breaks: a parser written as parseInt(headers["retry-after"], 10) against a provider that only ever sends integers returns NaN the first time it meets a date, and whatever your code does with NaN — retry immediately, wait forever, throw — is not what you wanted.

The header is also advisory. It tells you the earliest sensible retry; it does not promise the retry will succeed, and nothing stops the server from rate-limiting you again at exactly that moment. Treat it as a lower bound on your wait, never as a schedule.

Four formats for one idea

Across providers you will meet these shapes. Check your target’s current documentation rather than trusting a table anywhere, including this one — the header set is one of the fastest-moving parts of a provider’s surface.

• Delay in seconds. Retry-After: 20. The simple case, and the one everybody writes their parser for.
• An HTTP-date. Retry-After: Wed, 12 Aug 2026 09:41:00 GMT. Requires date parsing and, as discussed below, a decision about whose clock you trust.
• Vendor reset headers with duration strings.OpenAI’s rate-limit documentation describes x-ratelimit-reset-requests and x-ratelimit-reset-tokens alongside the corresponding -limit- and -remaining- headers, with values written as durations such as 1s or 6m0s rather than as bare integers. A parser that assumes seconds reads 6m0s as 6.
• Vendor reset headers with timestamps. Anthropic’s rate-limit documentation describes anthropic-ratelimit-requests-reset and anthropic-ratelimit-tokens-reset as RFC 3339 timestamps, alongside retry-after on a 429.

There is a fifth case that is easy to miss: no header at all. Some providers, and most self-hosted inference servers, return a bare 429 or 503 with nothing useful attached. Your policy has to have a defined behaviour for that, and it cannot be “retry immediately”.

The part that does not survive translation

The formats are an annoyance. The semantics are the real migration hazard, because two of these headers answer different questions and they are routinely treated as interchangeable.

Retry-After is addressed to you: come back after this long. A reset header is addressed to the bucket: the quota window refills at this moment. Those coincide when you are the only caller. They diverge exactly when it matters — under contention, where many callers share a limit.

Consider what happens if every client honours a reset timestamp precisely. All of them are told the same instant, all of them sleep until it, and all of them wake together and issue their retries in the same few milliseconds. The bucket refills and is immediately drained by the synchronised herd, most of the retries are rejected, and the next round of reset headers points at the next window — so the synchronisation persists instead of decaying. This is the classic thundering herd, and honouring the server’s own number exactly is what causes it.

The fix is to treat the server’s time as a floor and sample your actual wait above it. If the header says 20 seconds, wait 20 plus a uniform random draw over a window comparable to the delay itself. You have given up nothing — you were never going to succeed before 20 — and you have broken the correlation between callers.

The second divergence: providers meter more than one thing. A request can be limited by requests per minute or by tokens per minute, and the two buckets reset at different times. If you read only one reset header you will retry against the bucket that is still empty. Read both and take the later of the two.

A fallback ladder that works everywhere

Write one policy and one per-provider parser. The parser returns a delay and where it came from; the policy decides what to do with it. Keeping the provenance is worth the extra field, because “waited 60 seconds because the server said so” and “waited 60 seconds because we guessed” want different alerts.

function parseRetryAfter(headers, now = Date.now()) {
  const ra = headers.get("retry-after");
  if (ra) {
    if (/^\d+$/.test(ra.trim())) return { ms: Number(ra) * 1000, source: "retry-after-seconds" };
    const at = Date.parse(ra);
    if (!Number.isNaN(at)) return { ms: Math.max(0, at - now), source: "retry-after-date" };
  }
  // vendor reset headers: take the later of the request and token buckets
  const candidates = [];
  for (const name of ["x-ratelimit-reset-requests", "x-ratelimit-reset-tokens"]) {
    const v = headers.get(name);
    if (v) candidates.push(parseGoDuration(v));       // "6m0s" -> 360000
  }
  for (const name of ["anthropic-ratelimit-requests-reset", "anthropic-ratelimit-tokens-reset"]) {
    const v = headers.get(name);
    if (v) candidates.push(Math.max(0, Date.parse(v) - now));
  }
  const ms = candidates.filter((n) => Number.isFinite(n)).sort((a, b) => b - a)[0];
  return ms === undefined ? { ms: null, source: "none" } : { ms, source: "reset-header" };
}

function nextDelay(parsed, attempt, base = 500, cap = 60000) {
  const floor = parsed.ms ?? Math.min(cap, base * 2 ** attempt);
  const jitter = Math.random() * Math.min(floor, 5000);
  return Math.min(cap * 2, floor + jitter);
}

Three policy decisions sit on top of that function and are worth making explicitly rather than inheriting from a library default:

• Cap what you will honour. A provider entitled to say “retry in 3600 seconds” is entitled to park a worker for an hour. Above some threshold the right response is not to wait but to fail over to another binding, or to shed the request. The threshold is a product decision, not a networking one.
• Bound the total, not just each step. A retry budget per request — total elapsed, or total attempts — is what stops a degraded provider from converting into unbounded latency upstream.
• Do not retry across a provider boundary mid-request.The reasoning is in the in-flight requests page: a retry that changes provider is a new request, and during a cutover it is a double charge.

Three traps in the parser

Clock skew. Any date-based form requires you to subtract a remote timestamp from a local one, and if your host’s clock is a few seconds fast you compute a delay that is short by that much and retry into a still-closed window. The response’s own Date header is the server’s view of now; using it as the reference instead of your local clock removes the skew entirely, which costs one line and is worth it.

Header casing and access. HTTP field names are case-insensitive, but the object your SDK hands you may not be. If your old provider’s SDK returned a normalising Headers instance and the new one returns a plain object with whatever casing arrived on the wire, a lookup of headers["Retry-After"] starts returning undefined with no error. Normalise once at the adapter boundary.

Silent absence. The most damaging outcome is not a wrong delay; it is a missing header treated as zero. That converts a rate limit into a retry storm that makes the limit worse and can look, from the provider’s side, indistinguishable from abuse. Make the “no usable header” branch explicit, make it fall through to exponential backoff with full jitter, and count it — a sudden rise in that counter after a migration means your new provider is telling you something in a header you are not reading. The library’s test for 429 backoff is the place to pin the behaviour so the next SDK bump cannot quietly change it.