Testing a Timeout on a Stream That Never Sends Its Final Chunk

The request is open, the connection is healthy, bytes are arriving, and the answer never completes. Every timeout you configured is doing exactly what it was documented to do, and none of them will fire.

Three different timeouts

The word covers three unrelated quantities, and the reason streams hang is almost always that the one configured is not the one needed.

• Connect timeout. How long to wait for the TCP and TLS handshake. Irrelevant here: the connection succeeded.
• Read timeout. How long to wait between bytes. This is what most HTTP clients mean by a bare timeout parameter, and it is a per-read socket timeout rather than a total. Python’s httpx makes the distinction explicit by taking separate connect, read, write and pool values.
• Total deadline. How long the whole operation may take. Many clients do not offer one for streaming responses at all, and this is the one you actually need.

Set a read timeout of ten seconds on a stream that emits something every five, and the operation can legitimately run until the heat death of your process. The client is behaving correctly; the configuration describes a property nobody cared about.

There is a fourth quantity that is not a timeout at all and is often what people mean: time to first token. A stream that opens instantly and produces nothing for forty seconds is a different problem from one that streams smoothly and stalls at the end, and they want different budgets — a short one before the first content frame, a longer one between frames after that. Configuring a single value for both forces you to choose between cutting off a slow start and tolerating a long stall. Measure and assert them separately.

Why keepalives defeat the read timeout

Streaming endpoints send keepalives on purpose. Proxies and load balancers close idle connections, so a server that is thinking — running a long tool call, waiting on a slow upstream, queueing behind other work — emits a comment line or a ping event to keep the connection alive. In SSE that is a line beginning with a colon, which a conforming parser ignores.

That is the whole mechanism of the hang. Content stopped; traffic did not. Any timer that resets on received bytes will never expire, and a timer that resets only on content bytes will. So the client needs two clocks:

// Total deadline plus an idle clock that only content resets.
const IDLE_MS = 20_000;
const TOTAL_MS = 120_000;

async function readStream(url, body, onToken) {
  const total = AbortSignal.timeout(TOTAL_MS);
  const idle = new AbortController();
  let timer = setTimeout(() => idle.abort(new Error("idle")), IDLE_MS);

  const res = await fetch(url, {
    method: "POST",
    body: JSON.stringify(body),
    signal: AbortSignal.any([total, idle.signal]),
  });

  let sawTerminal = false;
  for await (const frame of parseSse(res.body)) {
    if (frame.comment) continue;          // keepalive: does NOT reset idle
    clearTimeout(timer);
    timer = setTimeout(() => idle.abort(new Error("idle")), IDLE_MS);
    if (frame.terminal) { sawTerminal = true; break; }
    onToken(frame);
  }
  clearTimeout(timer);

  if (!sawTerminal) throw new IncompleteStreamError();
}

AbortSignal.timeout and AbortSignal.any are standard and compose exactly for this: one signal that fires on either clock, passed once to fetch. In Python the equivalent is an httpx.Timeout with an explicit read value wrapped in an outer asynchronous timeout for the total.

The missing terminal frame

The second bug hides inside the first and is worse, because it produces no symptom at all. A stream that closes cleanly without its terminal frame — [DONE] in one dialect, a stop event in another — is an incomplete answer. A client that treats end-of-body as completion renders a half-finished response with no error, marks it done, caches it, and shows the user a sentence that stops mid-clause.

The invariant to assert is: the read loop exits either by seeing the terminal frame or by raising. There is no third exit. That single rule also covers the truncation case where the provider stops for its own reasons, which you can distinguish afterwards by finish reason — related to handling an empty completion.

Distinguish it from an intentional stop, too. A provider ending a stream because it hit a length limit sends a terminal frame with a finish reason; a stream that just ends sends nothing. Both leave you with a partial answer, but one is a documented outcome your application can explain to the user and the other is an unknown that should be retried. Collapsing them into a single “incomplete” state throws away the only information that distinguishes a bug from a budget.

A stub server that misbehaves on demand

You cannot ask a provider to hang, so the fixture is a local server with a mode switch. It is twenty lines and it removes every excuse for not testing these paths.

import { createServer } from "node:http";

export function hangingServer(mode) {
  return createServer((req, res) => {
    res.writeHead(200, { "content-type": "text/event-stream" });
    if (mode === "silent") return;                    // headers, then nothing

    res.write("data: " + JSON.stringify({ text: "The " }) + "\n\n");
    res.write("data: " + JSON.stringify({ text: "invoice " }) + "\n\n");

    if (mode === "keepalive-forever") {
      setInterval(() => res.write(": ping\n\n"), 1000);
      return;                                          // never terminates
    }
    if (mode === "closes-early") {
      res.end();                                       // no terminal frame
    }
  });
}

Keep the modes as separate fixtures rather than one server with branching logic driven by a header, so a test reads as its scenario. And bind it to port zero, letting the operating system choose, or a suite running in parallel fights itself over a hard-coded port and produces a class of flake indistinguishable from the bug you are testing for.

The four assertions

1. Silent server. Headers arrive, no data ever. Assert the idle timeout raises within its budget. Use fake timers so the test costs milliseconds; a suite with a real twenty-second wait gets skipped within a month.
2. Keepalives forever. Assert the operation aborts — this is the case that fails on almost every client written without thinking about it, and the assertion is on which error, idle or total, so the diagnosis in production is unambiguous.
3. Closes early. Assert an incomplete-stream error is raised, and separately that the two tokens already received are still available to the caller. Discarding partial output on a timeout is a second bug hiding behind the first, and users prefer a marked-partial answer to nothing.
4. The happy path is not slowed. A stream that finishes normally, well inside both budgets, must not be aborted and must clear its timers. A test that fills the process with orphaned timers passes and leaks; assert the completion path clears them, or run the suite with a handle checker.

One more assertion is worth adding at the application layer: that an aborted stream does not leave the request open on the server. When the client aborts, the underlying request should be cancelled, or you keep paying for tokens nobody will read — the same accounting problem described in streaming cost. Assert the fake server observed a client disconnect, not just that your promise rejected.