Testing Exponential Backoff on a 429 From an LLM Provider

A backoff test that actually sleeps is a test nobody runs. Two retries at one and two seconds is three seconds of wall clock for one case, and the suite that covers five cases takes a minute to tell you nothing new.

Assert the sequence, not the outcome

“It eventually succeeded” is already covered by the retry test. What is unique here is the shape of the waiting, and it is worth asserting because every part of it is a decision that can be wrong:

• The base and the growth. That the delays are 500ms, 1000ms, 2000ms and not 500ms three times — the bug you get from computing the delay from a variable that never increments.
• The first attempt is immediate. Sleeping before the first request adds latency to every single call in production for no benefit at all, and it is a common off-by-one in a loop written as “sleep, then try”.
• The ceiling holds. Unbounded doubling reaches absurd waits by the seventh attempt; a request that hangs for four minutes has usually already lost its user.
• A server-specified delay overrides the computed one. Covered below, and the one most often missing.

The fake clock, and the deadlock that catches everyone

Vitest replaces the timer functions with vi.useFakeTimers(), and time then advances only when you say so. The asynchronous advance helper is the one you want here, because the code under test awaits a promise that resolves from a timer; vi.advanceTimersByTimeAsync is documented as advancing timers including those scheduled asynchronously, and returns a promise you await. The synchronous advanceTimersByTime will not let the intervening microtasks run and your test will hang.

The second trap is ordering, and it catches nearly everyone once: you must start the operation and not await it, advance the clock, and only then await the result. Awaiting first means the test is blocked inside the sleep and never reaches the line that would have released it.

import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { withBackoff } from "../src/backoff";

beforeEach(() => vi.useFakeTimers());
afterEach(() => vi.useRealTimers());

it("waits 500ms, then 1000ms, and does not wait before the first attempt", async () => {
  const at: number[] = [];
  const start = Date.now();

  const operation = vi.fn(async () => {
    at.push(Date.now() - start);
    if (at.length < 3) {
      const err: any = new Error("Rate limit reached");
      err.status = 429;
      throw err;
    }
    return "ok";
  });

  // Start it, but do NOT await yet.
  const promise = withBackoff(operation, { baseMs: 500, maxAttempts: 3 });

  await vi.advanceTimersByTimeAsync(0);
  expect(operation).toHaveBeenCalledTimes(1);      // no sleep before attempt 1

  await vi.advanceTimersByTimeAsync(499);
  expect(operation).toHaveBeenCalledTimes(1);      // still waiting

  await vi.advanceTimersByTimeAsync(1);
  expect(operation).toHaveBeenCalledTimes(2);      // fired at exactly 500ms

  await vi.advanceTimersByTimeAsync(1000);
  expect(operation).toHaveBeenCalledTimes(3);

  await expect(promise).resolves.toBe("ok");
  expect(at).toEqual([0, 500, 1500]);
});

The pair of assertions at 499ms and 500ms is what makes this a real test of the delay rather than a test that some delay happened. Fake timers make that boundary check free, where with real sleeps it would be the flakiest assertion in the suite.

The fake clock has a second effect worth exploiting: it makes extra cases nearly free. A backoff implementation has a handful of parameters and the bugs live in their interactions, so write a case for each combination you actually ship — a base delay with and without a ceiling, two attempts and five, a server-specified delay present and absent — rather than one representative test. With real sleeps that suite would take minutes, somebody would mark it slow and it would be quietly excluded from the default run.

retry-after must win

A 429 from a provider usually carries a header telling you when to come back, and the whole point of that header is that the server knows something your exponential curve does not. The RFC 9110 definition of Retry-After allows either a number of seconds or an HTTP date, published by the IETF in June 2022 at rfc-editor.org, and providers additionally emit their own headers — OpenAI documents x-ratelimit-reset-requests and x-ratelimit-reset-tokens alongside the remaining counts.

So there are three tests here, and the third is the one that is usually missing:

it("honours retry-after in seconds instead of its own curve", async () => {
  server.use(
    http.post("https://api.openai.com/v1/chat/completions", () =>
      HttpResponse.json({ error: { message: "Rate limit reached" } }, {
        status: 429,
        headers: { "retry-after": "7" },
      }),
    { once: true }),
    http.post("https://api.openai.com/v1/chat/completions", () => okReply()),
  );

  const promise = classify("…");

  await vi.advanceTimersByTimeAsync(6_999);
  expect(attempts).toBe(1);
  await vi.advanceTimersByTimeAsync(1);
  await expect(promise).resolves.toBe("billing");
});

Then one test for the date form — a Retry-After of Wed, 12 Aug 2026 09:00:00 GMT parsed against a system time you pin with vi.setSystemTime — and one for a header that is absurd or malformed. A provider returning retry-after: 86400 during an incident should not make your service hang for a day: clamp it, and assert the clamp. A non-numeric header must fall back to the computed delay rather than producing NaN, which in most sleep implementations resolves immediately and turns your backoff into a hot loop against a service that just asked you to stop.

Inject the randomness, do not assert around it

Jitter is not optional in production. Without it, every client that got a 429 in the same second retries in the same later second, and the thundering herd re-forms at exactly the moment the provider is recovering. With it, your delays are no longer exact and the test above stops working.

The wrong fix is to loosen the assertion to a range, which weakens every test on the page. The right fix is to inject the random source, which is one extra parameter:

export type BackoffOptions = {
  baseMs: number;
  maxAttempts: number;
  maxDelayMs?: number;
  /** Injectable for tests. Defaults to Math.random. */
  random?: () => number;
};

// delay = min(maxDelayMs, base * 2 ** (attempt - 1)) * (0.5 + random() * 0.5)

Now pass random: () => 1 for the exact-sequence tests, random: () => 0 for a test that the floor is still non-negative and non-zero, and a seeded generator for a property test asserting that every delay across a thousand draws lands within the documented band and is non-decreasing in expectation. Determinism is restored without pretending jitter does not exist.

The ceiling and the deadline

The last two assertions are about the shape of the tail. First, that the delay stops doubling: with baseMs: 500 and maxDelayMs: 8000, the sequence should be 500, 1000, 2000, 4000, 8000, 8000 — assert the sixth, because that is the one that is wrong if the clamp is applied to the exponent rather than to the result.

Second, that there is an overall deadline and not only an attempt count. Five attempts with a growing backoff can exceed any request timeout your caller is willing to wait for, which means the user is gone before the last attempt is made and every attempt after that point is money spent on a response nobody will read. Assert that a total budget aborts the loop mid-wait — the interaction between that deadline and a per-call timeout is the subject of testing a timeout around a slow call.