Skip to content
Multigrid

Output Diff Viewer

Compare two model outputs word by word, choosing what should not count as a difference, and see how much of the change your normalisation absorbed.

Everything on this page runs in your browser. Nothing you paste is uploaded, logged, or written into the URL — only the settings are, so a configured tool can still be linked to.

What should not count as a differenceEach of these is applied before the comparison and undone before the display, so the diff below still shows your original text.
Tokens unchanged
97.1%

5 tokens differ.

{ "currency": "USD", "invoice_id": "INV-2201", "line_items": [ { "price": … 49 unchanged characters … { "price": 250, "qty": 1, "sku": "B-07" } ], "notes": "Net 30 — - paid in full. FULL", "total": 1250 }
Tokens compared, A / B
87 / 86
Tokens ignored by your settings, A / B
33 / 33
Runs removed from A
2
Runs added in B
2
Differences with no normalisation at all
5
Differences your normalisation absorbed
0
Compared as canonical JSON
yes
  • noteA, line 16, column 192 characters present in A and absent from B

    Expected: the same text on both sides, if these two runs were meant to agree

    … }⏎ ],⏎ "notes": "Net 30 — paid in full.",⏎ "total"…
  • noteB, line 16, column 192 characters present in B and absent from A

    Expected: the same text on both sides, if these two runs were meant to agree

    … }⏎ ],⏎ "notes": "Net 30 - paid in FULL",⏎ "total":…
  • noteA, line 16, column 296 characters present in A and absent from B

    Expected: the same text on both sides, if these two runs were meant to agree

    …"notes": "Net 30 — paid in full.",⏎ "total": 1250⏎}
  • noteB, line 16, column 295 characters present in B and absent from A

    Expected: the same text on both sides, if these two runs were meant to agree

    …"notes": "Net 30 - paid in FULL",⏎ "total": 1250⏎}
What this checked: this aligns the two texts with Myers' diff at the unit you chose, after applying the normalisations you ticked, and reports the position in the original text of every run that differs. It also runs the diff a second time with no normalisation at all, so the "differences absorbed" count is measured rather than assumed. It does not understand what the text means: two paraphrases that share no words are completely different here, and a correct answer worded differently from your reference scores zero. It does not check either side for validity beyond attempting a JSON parse, and it will not tell you which of the two outputs is better.
What this assumes: both sides are plain text in the same language and encoding. Alignment is capped at 6,000 tokens a side and 2,000 edits; past either the page says so rather than running to completion, because an unbounded diff of two unrelated documents is the same tab-freezing failure this whole family exists to avoid. Canonical JSON sorts keys with a plain code-unit sort and re-emits numbers through JavaScript, so 1250.00 and 1250.0 become the same token — and so would 1e3 and 1000.

Why the second number matters more than the first

Two runs of the same prompt almost never come back byte-identical, and the useful question is never "did anything change" — it is "did anything change that I care about". That is why this page runs the comparison twice. The raw count tells you how noisy the model is; the absorbed count tells you how much of that noise is formatting you were never going to act on. When those two numbers are close together, your regression suite is about to start failing for reasons that have nothing to do with quality.

The single most common finding is key order in a JSON output. Nothing in a language model guarantees field order, most schema validators do not care, and a plain string comparison in a test fails on every run. Turn on canonical JSON and the difference disappears — which is the answer, and also the argument for comparing parsed documents rather than strings in the test itself.

What a text diff cannot tell you

This measures surface form. "The invoice totals £1,250" and "Total: 1250 GBP" are the same fact and score as almost entirely different, while "approved" and "not approved" differ by one token and score as nearly identical. If you are grading correctness rather than stability, a diff is the wrong instrument — check the extracted fields, not the sentence. Where a diff is exactly right is the case it was built for: you changed one thing, and you want to see everything that moved as a result.

The reasoning behind this

Related tools

Output Diff Viewer · Multigrid