Skip to content
Multigrid

RAG Pipeline Simulator: Watch the Answer Fall Out of the Context

Mark the sentence that answers your question, then drag chunk size, overlap and k and watch whether that sentence survives chunking and makes it into the retrieved context.

The answer sentence, after chunking and retrieval
Present and intact

Chunk 3 holds the whole answer sentence and ranks #1 of 9, inside your top 2. The model will see it. Whether it uses it is a different question, and not one this page can answer.

orange band: the sentence you marked · solid bars: chunks retrieved · faint bars: chunks not sent
Words in the document
154
Chunks
9
Chunk size / overlap
24 / 6 words
Words stored across all chunks
202
Storage amplification from overlap
1.31×
Chunks holding the answer whole
3
Chunks holding part of it
4
Chunks retrieved
3, 4
Best rank achieved by any chunk containing the answer
#1
Characters of context assembled
282
Estimated context tokens (≈ chars ÷ 4)
71
Estimated input cost for one query
$0.00021
The context that would be sentPresent and intact
to one project and inherits that project's rate limit. A token that has not been used for ninety days is revoked automatically. A revoked

days is revoked automatically. A revoked token returns 401 on every request and cannot be reinstated; create a new one instead. Rate limits are
Where these numbers come from: This simulates retrieval, which is deterministic, and stops there. It does not show an answer, because producing one needs a model and this page has none — and a made-up answer that changed as you moved the sliders would be the most convincing dishonest thing on the site. What it can tell you is the half that a model cannot rescue: when the sentence containing the answer was cut in two, or was never sent. Every chunk boundary, every BM25 score and every rank above is computed from your own text.
What this assumes: Fixed-size chunking on whitespace-separated words with a sliding window, which is the default in most frameworks and the reason this failure is so common; sentence-aware and recursive splitters exist and would cut differently. Retrieval is BM25 at k₁ = 1.2, b = 0.75 over the chunks — no embeddings, no reranker, no query rewriting, each of which would change the ranking and none of which changes whether a split sentence can be reassembled. Tokens are estimated at four characters each, which is a rough English average and not your tokenizer; the price is a field you type. The chunk count is capped at 400.

The half of RAG that breaks quietly

A RAG pipeline has two failure modes and they are unequal. The visible one is the model getting the answer wrong when the right text was in front of it — annoying, but debuggable, because the evidence is in the prompt. The invisible one is the right text never arriving, and it produces exactly the same symptom: a confident, fluent, wrong answer. Teams spend weeks on prompt wording for problems that live entirely in the fifty lines of chunking code nobody reviewed.

Drag the chunk size through the point where the marked sentence straddles a boundary. The verdict changes to "cut in half" and stays there through a range of settings that look perfectly reasonable. Nothing errors. The retriever still returns k chunks, the model still answers, evaluation metrics that only measure whether a relevant chunk was retrieved still pass — and the sentence carrying the answer arrived as two fragments in two documents, one of which was ranked out. Overlap is the cheap mitigation, and you can watch it work: raise it until every boundary is covered twice, and watch the storage amplification row climb, because you pay to embed and store every duplicated word.

The other lesson is what k does. Raising it fixes rank problems and costs tokens on every single query for the lifetime of the product — the estimated cost row is per question, so multiply it by your traffic before deciding that top-10 is safer than top-3. And note what none of these controls can fix: if the answer needs two sentences from opposite ends of the document, no chunk size contains both. That is the case where retrieval is the wrong shape for the question, and no amount of tuning will tell you so — which is why the marked-span test above is worth running on your ten hardest questions before you tune anything.

The reasoning behind this

Related tools

RAG Pipeline Simulator: Watch the Answer Fall Out of the Context · Multigrid