Testing Your App's Behaviour Under a Simulated 429 From Every Provider

Rate-limit handling is written once, against one provider, and then quietly inherited by every provider added afterwards. The test that catches the inheritance being wrong is a single parameterised suite that replays each provider’s own 429 — status line, body shape and headers — and asserts your code behaves identically for all of them.

A 429 is not one thing

The status code is shared; almost nothing else is. OpenAI’s documented error envelope nests everything under an error object carrying message, type and code, and the rate-limit case is identified by the code rate_limit_exceeded. Anthropic’s documented envelope is a top-level "type": "error" alongside a nested error object whose own type is rate_limit_error.

That difference is the whole reason this test exists. Code that reaches for a field by name will get undefined on the other provider and fall through to a generic branch that honours no retry hint at all — and because the generic branch usually still retries on a bare status code, the behaviour looks approximately right in a smoke test and is wrong under load. Neither envelope is a variant of the other; they are two designs that happen to share a status line.

The headers diverge further. Some responses carry retry-after; some carry vendor-prefixed remaining-and-reset headers instead; some carry both; some carry neither, on the same endpoint, depending on which limiter fired. A handler written against one example is a handler written against one limiter.

One handler per provider

Mock at the network boundary rather than by stubbing the SDK client, because stubbing the client skips exactly the code you are testing — the SDK’s own retry layer, its error class mapping and its header parsing. In JavaScript, msw intercepts at that boundary; its handler API takes a matcher and a resolver, with a third options argument where once: true retires the handler after its first match.

// tests/rate-limit.handlers.ts
import { http, HttpResponse } from "msw";

export const rateLimited = {
  openai: http.post(
    "https://api.openai.com/v1/chat/completions",
    () =>
      HttpResponse.json(
        {
          error: {
            message: "Rate limit reached for gpt-4o in organization org-x.",
            type: "requests",
            code: "rate_limit_exceeded",
          },
        },
        { status: 429, headers: { "retry-after": "12" } },
      ),
    { once: true },
  ),

  anthropic: http.post(
    "https://api.anthropic.com/v1/messages",
    () =>
      HttpResponse.json(
        {
          type: "error",
          error: { type: "rate_limit_error", message: "Number of requests has exceeded your rate limit." },
        },
        { status: 429, headers: { "retry-after": "12" } },
      ),
    { once: true },
  ),
};

The once: true is load-bearing. Without it the handler answers every attempt, your retry loop never succeeds, and the test measures your retry ceiling instead of your recovery. With it, attempt one gets the 429 and attempt two falls through to the success handler, which is the sequence a real limiter produces.

What to assert

Not “it did not throw”. The point of running the same scenario across every provider is that the observable behaviour should be identical, so assert on the things that are supposed to be the same:

• One error type escapes. Whatever your normalised rate-limit error is called, every provider’s 429 that exhausts retries surfaces as that type — not as a raw SDK error from one provider and a normalised one from another.
• The retry count matches the policy. Assert the number of outbound requests, not the elapsed time. Count them by incrementing in the resolver, and compare against the configured maximum plus one.
• The delay honoured the hint. Under fake timers, assert the scheduled delay was at least the value in retry-after rather than your default backoff. A handler that ignores the header and sleeps 200 ms will be limited again immediately.
• The eventual success returns a normal result. A surprising number of retry wrappers succeed on the retry and then return the retry’s raw body instead of the parsed one.
• Nothing was double-charged. If your code records usage or cost per attempt, assert the failed attempt contributed zero, since a 429 has no usage.

import { describe, expect, it } from "vitest";
import { rateLimited } from "./rate-limit.handlers";
import { server } from "./msw-server";
import { complete, RateLimitedError } from "../src/llm";

describe.each(Object.keys(rateLimited))("429 from %s", (provider) => {
  it("retries once and surfaces one normalised error type when it keeps failing", async () => {
    let attempts = 0;
    server.use(rateLimited[provider]);
    server.events.on("request:start", () => { attempts += 1; });

    await expect(complete({ provider, prompt: "hi", maxRetries: 1 }))
      .rejects.toBeInstanceOf(RateLimitedError);
    expect(attempts).toBe(2);
  });
});

Not every 429 should be retried

This is the assertion the harness exists for and the one most suites are missing. A 429 can mean “you are going too fast, slow down” or it can mean “your account has no credit and this will never succeed”. Retrying the second forever is how a bad card turns into an hour of pointless load and a support ticket. OpenAI’s error reference distinguishes the quota case with the type insufficient_quota, which arrives with the same 429 status as an ordinary rate limit.

So add a case per provider whose expected behaviour is the opposite: exactly one request, no retries, and a distinct non-retryable error class. If your classifier keys only on the status code, this case will fail immediately, which is the point.

Retry-After has two formats

RFC 9110 defines Retry-After as either a number of seconds or an HTTP-date. Providers overwhelmingly send seconds, and a parser that assumes seconds will read the date form as NaN and usually fall back to zero — an immediate retry into an active limiter. Add one case per format to your header parser’s own unit tests: a plain integer, a fractional value, an HTTP-date in the future, an HTTP-date in the past, an absent header, and a garbage value. All six should produce a finite, non-negative delay, and the garbage case should produce your default rather than zero. Then clamp: a provider returning a very large value should not park your worker for an hour — cap it and fail over instead, which is what a fallback-order test exercises.