Streaming Structured Output: Parsing Incomplete JSON

Streaming a structured response is only useful if you can render it before it ends, and no prefix of a JSON document is a JSON document. The fix is about eighty lines and it is worth writing rather than installing.

Why a prefix is not JSON

Halfway through a stream your buffer looks like {"items":[{"name":"wid. JSON.parse throws, so the naive implementation is to parse on every chunk, swallow the exception, and render nothing until the last token arrives — which is streaming with none of the benefit.

The three states a prefix can be in are all recoverable. You may be inside an unterminated string; you may be inside one or more unterminated containers; you may be mid-way through a literal such as tru or 12.. The last one is the subtle case, because 12 at the end of the buffer is not necessarily the number 12 — the next chunk may make it 125. Any parser that does not treat a trailing literal as unfinished will render numbers that change under the reader.

Truncate to safety, then close

The approach is a single left-to-right scan that tracks two things: the stack of open containers, and the last offset at which the document was in a state that can be legally closed. Then emit the buffer truncated to that offset, plus the closing brackets the stack owes. No backtracking, no repeated parse attempts, one pass per chunk.

The one non-obvious rule: the closing quote of an object key is not a safe truncation point. Truncating at {"name" and closing gives {"name"}, which is not valid JSON. The closing quote of a value is safe. So the scanner has to know which slot of an object it is in.

The parser

// Returns the largest valid JSON value contained in a prefix, or undefined.
// Never throws. Safe to call on every streamed chunk.

function scanString(s, i) {
  // i points at the opening quote. Index just past the closing quote, or -1.
  for (let j = i + 1; j < s.length; j++) {
    if (s[j] === "\\") { j++; continue; }
    if (s[j] === '"') return j + 1;
  }
  return -1;
}

const LITERAL = /^(-?\d+(?:\.\d+)?(?:[eE][+-]?\d+)?|true|false|null)/;

export function parsePartialJson(raw) {
  const s = raw.replace(/^\s*```(?:json)?\s*/i, "").trimEnd();
  const stack = [];          // { kind: "obj" | "arr", slot }
  let safeEnd = -1;
  let safeStack = null;
  const mark = (end) => { safeEnd = end; safeStack = stack.map((f) => f.kind); };

  let i = 0;
  while (i < s.length) {
    const ch = s[i];
    if (ch === " " || ch === "\n" || ch === "\t" || ch === "\r") { i++; continue; }
    const top = stack[stack.length - 1];

    if (ch === "{" || ch === "[") {
      stack.push({ kind: ch === "{" ? "obj" : "arr",
                   slot: ch === "{" ? "key" : "value" });
      mark(i + 1); i++; continue;
    }
    if (ch === "}" || ch === "]") {
      stack.pop();
      const parent = stack[stack.length - 1];
      if (parent) parent.slot = "after";
      mark(i + 1); i++; continue;
    }
    if (ch === ":") { if (top) top.slot = "value"; i++; continue; }
    if (ch === ",") {
      if (top) top.slot = top.kind === "obj" ? "key" : "value";
      i++; continue;
    }
    if (ch === '"') {
      const end = scanString(s, i);
      if (end < 0) break;                       // string still streaming
      if (top && top.kind === "obj" && top.slot === "key") top.slot = "colon";
      else { if (top) top.slot = "after"; mark(end); }   // a VALUE is a safe end
      i = end; continue;
    }
    const m = LITERAL.exec(s.slice(i));
    if (!m) break;                              // garbage; keep what we have
    const end = i + m[0].length;
    if (end === s.length) break;                // "12" may still become "125"
    if (top) top.slot = "after";
    mark(end); i = end;
  }

  if (safeEnd < 0) return undefined;
  let out = s.slice(0, safeEnd);
  for (let k = safeStack.length - 1; k >= 0; k--) {
    out += safeStack[k] === "obj" ? "}" : "]";
  }
  try { return JSON.parse(out); } catch { return undefined; }
}

The property worth testing, and the one that makes this trustworthy, is that it holds for every prefix rather than the handful you thought of. Sweep it:

const doc = '{"items":[{"name":"widget","qty":3,"tags":["a","b"]}],"total":15}';
for (let n = 0; n <= doc.length; n++) {
  const v = parsePartialJson(doc.slice(0, n));
  // v is undefined or a real value; it never throws, and JSON.parse
  // already succeeded inside the function, so v round-trips.
}

What it does on real prefixes

The {"a":true case shows the trade deliberately. A trailing literal is treated as still growing, because the parser cannot distinguish true from a prefix of something longer without lookahead it does not have. Booleans and null could be special-cased since no longer literal starts with them; numbers cannot. Leaving all three conservative keeps one rule instead of three, and the field appears one chunk later.

Using it in a UI without flicker

• Render fields, not the object. Bind each field separately and leave it absent until it appears. Re-rendering a whole form because one key arrived is what makes streaming UIs feel broken.
• Never shrink. An array that showed three items must not show two on the next chunk. Merge into your previous state by index rather than replacing it; the parser is monotonic in what it can see but your renderer should enforce it anyway.
• Do not act on partial values. Display them; do not validate them, do not submit them, do not fire a request off them. "wid" is not a product name.
• Throttle. Parsing on every SSE chunk is fine — one pass over a few KB — but re-rendering on every chunk is not. A 60ms trailing debounce is invisible to a reader and removes most of the work.
• Validate once, at the end. The partial parser is a rendering aid. The real record is the one that arrives complete and passes your schema.

One protocol note: if you are streaming a tool call rather than a message, the JSON arrives in a different field — deltas accumulate into the tool call’s arguments string, one buffer per tool call index, and you run the same parser over that buffer. Parallel tool calls arrive interleaved, so key the buffers by the delta’s index rather than appending everything to one string; that mistake produces a single corrupted argument object and is hard to spot because it only manifests when the model happens to call two tools.

A note on why this is eighty lines of your own code rather than a dependency. Streaming JSON libraries in the SAX tradition are built for a different problem — a valid document too large for memory — and they raise on a truncated one, which is the only case you have. Repair-style libraries exist and mostly work by attempting a parse, mutating the string on failure, and retrying, which is quadratic in the worst case when you are calling it on every chunk of a growing buffer. The scan above is one pass, allocates one small array, and has behaviour you can state in a sentence: it returns the largest valid prefix value or nothing, and it never throws. Predictability is worth more than coverage here, because this code runs sixty times a second in a render loop.