Fixing a Streaming Response Buffered Instead of Chunked on Vercel

The code streams. Locally it streams. Deployed, the browser sits blank for nine seconds and then paints the entire answer at once. Something between your ReadableStream and the reader is collecting the whole body before forwarding it, and there are only four places it can be.

The symptom, precisely

Buffering is not an error. Nothing fails, no status code changes, no log line appears. The response is byte-for-byte correct and arrives in one piece at the end, which is why it survives review and is discovered by a user saying the app feels slow.

It is also not the same as slow generation, and confusing the two wastes the first hour. The distinguishing measurement is time to first byte against total time. A working stream has a small first-byte time and a much larger total. A buffered stream has the two within milliseconds of each other, whatever the absolute numbers are.

Bisect before you change anything

Four layers can buffer: your function, the platform in front of it, an intermediary, and the client. Test them in order and you will identify the layer in about five minutes; guess at headers and you can spend a day.

1. Take the browser out. Call the deployed endpoint with curl -N, which disables curl’s own output buffering, and ask for the two timings:

   curl -N -X POST https://your-app.vercel.app/api/chat \
     -H 'Content-Type: application/json' \
     -d '{"prompt":"Count slowly from one to thirty."}' \
     -w '\nfirst_byte: %{time_starttransfer}s  total: %{time_total}s\n'

   If text appears progressively and first_byte is far below total, the server is fine and the problem is in your client. Stop here and go to the last section.
2. Compare against the provider directly. Run the equivalent curl -N against the provider’s own streaming endpoint from the same machine. If that also arrives in one piece, your function was never receiving a stream — check that you passed stream: true and that you are not awaiting the whole body.
3. Inspect the response headers. curl -sI will not do — a HEAD request can behave differently — so read them from the -N run with -D -. You are looking for the content type, and for a Content-Length, which is proof of buffering: a length can only be known by something that has the whole body.
4. Check where the function lives. See the next section; this is the cause most often missed entirely.

Causes on the server

• The function is in the wrong directory. Vercel states it directly in its streaming documentation: streaming functions must be defined in an app directory, even if the rest of your application is in the pages directory. A handler in pages/api is not a streaming function, and it fails by buffering rather than by erroring. This is the single most common cause and no amount of header tuning will fix it.
• The stream is being consumed before it is returned. await upstream.json() and await upstream.text() both resolve only when the body ends. So does awaiting a helper that resolves to a complete string. If you are not passing a ReadableStream or an object wrapping one into the Response constructor, nothing downstream can stream:

  // buffers — waits for the whole completion
  const data = await upstream.json();
  return Response.json(data);
  
  // streams — forwards chunks as they arrive
  return new Response(upstream.body, {
    headers: {
      "Content-Type": "text/event-stream; charset=utf-8",
      "Cache-Control": "no-cache, no-transform",
    },
  });

• The content type invites collection. Vercel’s own streaming example sets Content-Type: text/event-stream explicitly. An application/json body is something a client and an intermediary both have every reason to parse whole; an event stream is one they know to forward.
• A transform that accumulates. A TransformStream that appends to a buffer waiting for a complete JSON object is a correct-looking piece of code that reintroduces buffering inside your own function. If you must reassemble across chunk boundaries, emit on every completed line rather than at the end.

Causes on the wire

• Compression. A compressor that must see the whole body to produce a Content-Encoding destroys streaming by construction. Cache-Control: no-transform is the standard request to intermediaries not to recompress or otherwise rewrite the body, and is worth setting on any streamed response.
• A reverse proxy in front of Vercel. If your domain points at an nginx or an nginx-derived layer before it reaches Vercel, proxy response buffering is on by default there. nginx documents X-Accel-Buffering: no as the response header that disables it per response — see the nginx proxy_buffering documentation. It is harmless to send when nothing is listening for it, which makes it a reasonable default on streamed responses.
• Anything that reads the body in the middle. Proxy or middleware code that inspects or rewrites a response body has to hold it. Middleware should pass a streamed response through untouched.
• A cache in the path. A CDN storing a response must have all of it. Cache-Control: no-cache on a streamed, per-user completion is correct on its own merits and removes this as a possibility.

One Vercel-specific deadline interacts with all of this: functions using the Edge runtime must begin sending a response within 25 seconds, and may then stream for up to 300. Buffering converts a healthy stream into a single late write, so a stream that would have started at second two now starts whenever generation finishes — and if that is past 25 seconds, a buffering bug presents as a timeout rather than as slowness. See the invocation timeout page if that is what you are actually looking at.

Causes in the client

If curl -N streams and the browser does not, the server is correct and every server-side change from here is wasted effort.

• Awaiting the whole body. await res.text(), await res.json() and any HTTP client that resolves with a complete body — which includes most promise-based wrappers in their default configuration — resolve at the end of the stream. Read response.body with a reader instead:

  const res = await fetch("/api/chat", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
  });
  
  const reader = res.body!.pipeThrough(new TextDecoderStream()).getReader();
  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    append(value);
  }

• Rendering only at the end. A framework that batches state updates can collect thousands of small appends into one paint. The stream arrived incrementally; the pixels did not. Check whether each chunk actually triggers a render before blaming the network.
• Devtools lying to you. The network panel commonly shows a response body only once complete, which makes a working stream look buffered. Trust the curl -N result over the panel.
• A service worker in the path. A worker that responds with a constructed Response built from an awaited body reintroduces buffering entirely inside the browser, after every server fix has landed.