Skip to content
Multigrid

Context Degradation Demo: Build the Needle Test

Generates a needle-in-a-haystack prompt at any depth, prices the full sweep, and plots the accuracy curve you measure on your own model.

The needle lands at roughly this token
~7,988

Of an estimated 16,079-token prompt: 7,988 tokens of filler before it, 8,018 after. One run costs about $0.05 in input at the price you gave.

Filler records generated
792
Filler tokens before the needle
7,988
The needle itself
18 tokens
Filler tokens after the needle
8,018
Estimated prompt size
16,079 tokens
Input cost per run
$0.05
Full 11-depth sweep × 5 trials
$2.65
Your results — click a cell once per successful retrieval

0% … 100% depth, left to right. Each click adds one success out of 5; clicking past 5 clears the cell.

What this assumes: nothing here calls a model, so nothing here produces an accuracy figure — the curve is yours, plotted from results you ran and typed in. Token counts are estimated at four characters per token and will differ from your tokenizer’s by a few percent; the depth percentage is a position in the filler, not a guaranteed token offset. The filler is synthetic log records with a fixed seed, so the same haystack size always produces the same haystack. A single needle at a single depth measures one point of one model on one kind of fact; published needle tests vary the fact type, the distractor content and the question form, and the shape of the curve moves when they do.

Run the test, don’t read about it

“Lost in the middle” has become a thing people cite rather than a thing people check, which is unfortunate, because it is one of the cheapest experiments in the entire field and the result is specific to the model you are about to ship. The shape it names — retrieval accuracy highest near the start and end of a long context, sagging in the middle — is a property of particular models at particular context lengths, not a law, and it is not safe to assume it transfers to the model in your production config at the length you actually send. That is a question with a two-hour answer, and this page is the two hours.

This page builds the experiment. The depth slider positions a fact nobody could know among filler that contains nothing relevant, the token accounting tells you what each run costs before you start, and the grid takes the results as you get them. Eleven depths at five trials is 55 calls; at the illustrative price and 16,000-token haystack above, the whole sweep is small change, and you will have a curve for your own stack instead of a citation.

Two things to be careful about when you run it. The needle must be genuinely unguessable, or the model answers from what it already knows and you measure nothing; a made-up passphrase is a good shape. And the filler must be plausible enough to actually be read but must not contain a near-miss for the question, or you are measuring distractor resistance instead of position.

If the curve is flat, long context is working for your case and you can stop worrying about ordering. If it sags, the fix is not a bigger window — it is retrieving less and putting it near the end.

The reasoning behind this

Related tools

Context Degradation Demo: Build the Needle Test · Multigrid