Streaming UX: Why Watching Text Appear Feels Faster

Streaming is the single largest perceived-latency win available in an AI product, and it does not make anything faster. Understanding precisely which quantity it improves is what tells you when it helps, when it is neutral, and when it actively makes the experience worse.

Streaming does not make it faster

Total wall-clock time to the last token is unchanged by streaming, and in some stacks marginally worse: chunked delivery adds framing overhead, and rendering incrementally costs more work on the client than rendering once. What streaming changes is when feedback starts — from “end of generation” to “time to first token”.

Those are two genuinely different numbers with different causes. Time to first token is dominated by prefill and grows with prompt length; the rate afterwards is roughly flat for a given model on given hardware. So streaming converts a wait of ttft + output/rate into a wait of ttft followed by a period in which something is visibly happening. For a 300-token answer that can be the difference between fifteen seconds of nothing and two seconds of nothing.

What the perceived-performance work found

The relevant literature predates language models by decades and is worth citing precisely, because “research shows streaming feels faster” is exactly the kind of claim that gets repeated without anyone checking what was actually studied.

• Myers (1985), CHI — the foundational study on percent-done progress indicators. It established that people prefer interfaces with progress indicators and find waiting with one less unpleasant, which is why progress indication became a default rather than an option.
• Harrison, Amento, Kuznetsov and Bell (2007), UIST — Rethinking the Progress Bar. The finding that matters here is that the same elapsed duration is perceived differently depending on the behaviour of the indicator: certain animation profiles make an identical wait feel shorter. Perceived duration is manipulable independently of actual duration.
• Harrison, Yeo and Hudson (2010), UIST — Faster Progress Bars, extending the above to the shape of the progress function over time, with the pacing profile again affecting perceived speed at fixed real duration.
• Buell and Norton (2011) — the labour illusion: showing the work being done can increase perceived value and tolerance of a wait, relative to delivering the same result instantly with no visible effort.

What transfers, and what does not

It is tempting to take the whole body of work across. Most of it does not apply directly, and being clear about that is the difference between a design decision and cargo cult.

There is one property of token streams with no analogue in the progress bar literature at all: the streamed content is the answer, not a proxy for progress. That is what makes it uniquely good — the user starts reading rather than waiting — and it is also what creates every problem in the rest of this page.

Jitter, reading speed and the smoothing buffer

Tokens do not arrive at a steady rate. They arrive in network-shaped bursts, with gaps at tool calls and at buffer boundaries. Rendered naively, this produces text that lurches — a paragraph in one frame, then nothing for two seconds — which reads as instability even when total throughput is good.

The fix is a smoothing buffer: accumulate arriving tokens and emit them to the DOM at a controlled rate slightly below the average arrival rate, so the display drains continuously rather than in jumps.

// Emits characters at a steady rate from a bursty source.
// Drains faster when the queue is long so it can never fall behind.
function createSmoother(onChunk, targetCharsPerSecond = 220) {
  let queue = "";
  let done = false;
  let last = performance.now();

  function tick(now) {
    const dt = (now - last) / 1000;
    last = now;

    // Catch-up term: a long queue drains proportionally faster,
    // so smoothing adds latency but never unbounded latency.
    const rate = targetCharsPerSecond * (1 + queue.length / 400);
    const take = Math.min(queue.length, Math.ceil(rate * dt));

    if (take > 0) {
      onChunk(queue.slice(0, take));
      queue = queue.slice(take);
    }
    if (!done || queue.length > 0) requestAnimationFrame(tick);
  }
  requestAnimationFrame(tick);

  return {
    push: (text) => { queue += text; },
    end:  () => { done = true; },
  };
}

The rate to target is bounded on both sides by the reader. Adult silent reading of ordinary prose sits in the low hundreds of words per minute; a few hundred characters per second is comfortably above that, so smoothing at that rate is imperceptible as a delay while removing the lurch. Below reading speed and the interface becomes the bottleneck — which is the case worth checking, because the fix there is a faster model or a shorter answer, not a UI change.

When not to stream

Streaming is a default, not a law, and there are cases where it is the wrong call for reasons that come straight from the mechanics.

• When output must be validated before display. A guardrail, a redaction pass or a schema check cannot run on text that has already been rendered. If you must not show unvalidated content, you must buffer, and buffering forfeits the entire win — that is the trade and it should be made explicitly rather than discovered.
• When the output is structured, not prose. A half-parsed JSON object is not readable progress, it is noise. Either stream at the field level once each field completes — see incremental parsing of partial JSON — or wait and render the finished object.
• When the answer can contradict itself mid-flight. A model that opens with a wrong figure and corrects it three sentences later has, in a streaming interface, shown the user a wrong figure. Some readers stop at the first number.
• When nobody is watching. A batch job, a webhook, a background summarisation. Streaming to a consumer that only uses the final string adds complexity and buys nothing.

The second of those is worth dwelling on, because it is where most streaming implementations quietly go wrong. Streaming is a commitment to render whatever arrives, and a commitment to render is a commitment not to inspect. Any requirement that begins “before showing the user, we must…” is incompatible with it. Teams usually discover this after shipping, when a compliance requirement arrives and the only options are to buffer everything — losing the win — or to redact text that is already on screen, which does not work and looks worse than never having shown it.

A middle path exists and is underused: buffer to a boundary rather than to the end. Hold each sentence or each paragraph, run the cheap checks on it, then release. The cost is one boundary of added latency instead of the entire generation, and the user still starts reading within a second or two of the first token. It also composes with the smoother above, since both are already queueing text on its way to the DOM — the check simply runs at the point where the queue drains.