Testing a Client's Behaviour When a Stream Drops Mid-Response

The failure this test exists for is not an exception. It is a user being shown half an answer, formatted normally, with no indication that anything went wrong — because the code treated “the stream ended” as “the model finished”.

Two kinds of short answer

A response can be shorter than expected for two completely different reasons, and they need opposite handling.

A complete stream reporting truncation. The model hit max_tokens, so the provider emits a final chunk with finish_reason: "length" (or Anthropic’s stop_reason: "max_tokens"), then its terminal frame, then closes. The transport worked perfectly. The correct response is to continue the generation or raise the limit, and the tokens are billed because they were produced.

An incomplete stream. The socket closed before the terminal frame. Nothing tells you why: a load balancer idle timeout, an upstream restart, a proxy’s response size cap, a mobile network handover. The correct response is to retry — and it is a different decision from the first case, because retrying a length truncation just burns the same tokens again. See what a retry costs.

Code that only looks at the accumulated text cannot tell these apart, and code that only checks for a thrown exception often cannot either, because in several runtimes the end of a truncated chunked response surfaces as an ordinary end-of-stream rather than an error.

The invariant that catches it

It is one sentence, and it is the whole page: a stream that ends without its terminal frame is an error, no matter how much content arrived first. Not a warning, not a partial success — an error, raised to the caller, with the partial text attached so a UI can choose to show it as explicitly incomplete.

This is why the terminal frame matters so much and why the clean-close test is its necessary partner. Together they say: the terminal frame is present on success and its absence is detected on failure. Either test alone can be satisfied by code that never looks at the frame at all.

Implementing it means your reader needs a state flag, not just an accumulator:

class IncompleteStreamError extends Error {
  constructor(readonly partial: string, readonly bytes: number) {
    super("stream ended after " + bytes + " bytes without a terminal frame");
    this.name = "IncompleteStreamError";
  }
}

async function readCompletion(res: Response): Promise<string> {
  let text = "", bytes = 0, sawTerminal = false;
  // ...framing loop, setting sawTerminal on the [DONE] line...
  if (!sawTerminal) throw new IncompleteStreamError(text, bytes);
  return text;
}

Making a connection drop

You need a server that writes some frames and then destroys the socket without a normal close. A Node HTTP server does this in one line, and unlike a mock at the fetch layer it produces a genuine transport-level truncation, which is the thing you are testing for.

import { createServer } from "node:http";
import { once } from "node:events";

async function truncatingServer(framesBeforeDrop: number) {
  const server = createServer((req, res) => {
    res.writeHead(200, {
      "content-type": "text/event-stream",
      "cache-control": "no-cache",
    });
    for (let i = 0; i < framesBeforeDrop; i++) {
      res.write('data: ' + JSON.stringify({
        choices: [{ index: 0, delta: { content: "word" + i + " " }, finish_reason: null }],
      }) + '\n\n');
    }
    // No terminal frame, no res.end(): kill the socket underneath it.
    res.socket!.destroy();
  });
  server.listen(0);
  await once(server, "listening");
  const { port } = server.address() as { port: number };
  return { url: "http://127.0.0.1:" + port, close: () => server.close() };
}

res.socket.destroy() rather than res.end() is the point. res.end() produces a valid, complete HTTP response that simply has no terminal frame — useful as a second test case, and a weaker one. destroy() aborts the chunked encoding mid-flight, which is what a real network failure looks like and what makes the client library raise.

It is worth knowing where the real ones come from, because it tells you which variants are worth writing. Load balancers and reverse proxies apply an idle timeout to a connection that has sent nothing for some seconds, and a model thinking before it answers, or working through a long tool call, can easily be idle for that long — which is what keepalive comments exist to prevent. Serverless platforms apply a maximum response duration. A deploy rolls the process holding the connection. And a mobile client changing network drops it outright. Only the last of those looks like a client problem; the rest are infrastructure you own, and all of them present downstream as this one test case.

What the error actually looks like

Both cases must be handled, and their shapes differ, so assert on both rather than on a message string. In Node with the built-in fetch, a socket destroyed mid-body typically surfaces as a TypeError whose message is terminated, with the real detail on err.cause — commonly a socket error with code: "ECONNRESET" or an undici-specific code. In the browser it is a TypeError with a vaguer message, by design. Under a clean res.end() with no terminal frame there is no exception at all, and only your own check fires.

So the assertion is about your error type, not theirs:

• The call rejects, and with IncompleteStreamError — not with a raw TypeError that leaked through, and not with a resolved value.
• err.partial contains the text that did arrive, in order. Do not throw it away; a UI that can show “the connection dropped, here is what we got” is materially better than one that shows nothing, and a retry policy may be able to continue from it.
• The error is distinguishable from a cancellation. An AbortError raised because the user pressed stop must not be reported as a network failure, and must not be retried. That distinction is the subject of the cancellation test.
• Nothing was left open: the same handle counter used in the clean-close test returns to zero on this path too.

The test

1. Start the truncating server with, say, three frames. Point your client at it.
2. Assert the promise rejects with your own error type: await expect(readCompletion(res)).rejects.toBeInstanceOf(IncompleteStreamError).
3. Capture the error and assert err.partial ends with word2 — that the frames received before the drop were not lost by the error path.
4. Add the second variant with a clean res.end() and no terminal frame. Assert the same rejection. If this one passes and the first fails, your code is relying on a transport exception rather than on the invariant.
5. Add a control case: a stream that ends with finish_reason: "length" and a proper terminal frame must not reject. It must resolve, and expose the finish reason so a caller can decide whether to continue. Without this case, a reader that rejects on any short answer passes every other assertion here.
6. Finally, assert the retry policy: the incomplete stream is retried and the length truncation is not. If the request had side effects — a tool the model already called, a row you already wrote — assert the retry carries the same idempotency key, so a drop that happened after the work was done does not do the work twice.

One thing not to do: retry inside the reader. It is tempting to have the framing loop reconnect and continue transparently, and it makes the bug much harder to see, because a stream that drops every time now presents as a slow one. Raise the error, let a policy above decide, and count the occurrences — a drop rate that climbs is a signal about your infrastructure that a transparent retry deletes.