Skip to content
Multigrid

Retrieval Debugger

Paste a query and your chunks to see a BM25 ranking broken down term by term, including the query terms that appear nowhere at all.

Top result
chunk 2

BM25 score 1.686, ahead of chunk 1 at 1.263. To cancel a subscription, open Settings, choose Billing, and select End plan. Access conti…

rankchunkscorelengthterms matched
1#2 To cancel a subscription, open Settings, choose Billing, and select En…1.686212 of 8
2#1 Refunds are issued to the original payment method within 5 to 10 busin…1.263161 of 8
3#4 If a charge appears twice, the duplicate is voided automatically withi…0.861201 of 8
4#3 Our uptime commitment is 99.9% per calendar month, measured at the edg…0.000200 of 8
5#5 Seats can be added at any time and are billed pro rata for the remaind…0.000190 of 8
chunk 1, term by termin chunkIDFcontribution
to20.8751.263
how02.4850.000
long02.4850.000
does02.4850.000
a00.8750.000
refund02.4850.000
take02.4850.000
arrive02.4850.000

Score 1.263 = the sum of the contributions. Length 16 tokens against an average of 19.2, giving a normalisation factor of 0.875 — above 1 means this chunk is penalised for being long.

Chunks parsed
5
Distinct query terms
8
Query terms found in no chunk at all
6
Average chunk length
19.2 tokens
Scoring
BM25, k₁ 1.20, b 0.75
  • errorquery term "how"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • errorquery term "long"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • errorquery term "does"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • errorquery term "refund"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • errorquery term "take"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • errorquery term "arrive"does not appear in any chunk, so it contributes nothing to any score

    Expected: the term, a stem of it, or a synonym present in the text — this is a vocabulary mismatch, and it is the single most common reason a lexical retriever misses an obviously relevant chunk

  • notechunk 1ranks 2 of 5, and contains none of: how, long, does, a, refund, take, arrive

    Expected: if you expected this chunk first, the terms above are what it would need — or a retriever that is not lexical

What this checked: this ranks the chunks you pasted against your query with BM25, and shows every term's IDF, in-chunk frequency and score contribution, so the ranking is fully reconstructable by hand. This is not your vector retriever. No embedding model runs on this page, so what you are debugging here is the lexical half of the problem: whether the words are there. That is worth checking first because vocabulary mismatch — the query saying "refund" where the document says "reimbursement" — is the most common cause of a retriever missing the obvious chunk, and it is invisible in a cosine score. What this cannot tell you: how your embedding model ranks these, how your reranker would reorder them, or how your production index behaves at scale.
What this assumes: terms are lowercase runs of letters and digits, so punctuation splits and no stemming happens — "refund" and "refunds" are different terms here, which is itself a useful thing to see, since a lexical index without a stemmer behaves exactly this way. IDF is the non-negative Lucene variant, ln(1 + (N − df + 0.5)/(df + 0.5)). Document length is counted in these terms, not in tokens from your model's tokenizer. With only a handful of chunks pasted, IDF values are computed over that handful and will differ from the ones your real index computes over the whole corpus — the ordering is usually stable, the absolute scores are not.

Everything on this page runs in your browser. Nothing you paste is uploaded, logged, or put in the URL — only the settings above the input are, so a configured tool can be linked to.

Start every retrieval bug here

When a RAG system returns the wrong chunk, the instinct is to blame the embedding model and start shopping for a better one. Check the words first. Paste the query and the chunk you expected into this page: if a query term appears in no chunk at all, the fix is a synonym, a rewrite step or a hybrid index, and no amount of embedding quality will make up for a corpus that uses different vocabulary from your users.

The other frequent culprit is length. BM25 penalises long documents because a long document has more chances to contain any term by accident, and the normalisation factor in the table above shows exactly how much your chunk is being marked down. Vector retrievers have their own version of this: a long chunk's embedding is an average of everything in it, so a single relevant sentence gets diluted by the nine irrelevant ones around it. Different mechanism, same symptom, same fix — smaller chunks.

Why lexical scoring is still worth having

Embeddings are bad at exactly the things BM25 is good at: rare tokens, product codes, error strings, names, numbers. A query for ERR_MODULE_NOT_FOUND or an invoice number needs an exact match, and a dense retriever will happily return five chunks about errors in general. That is why hybrid retrieval — BM25 and vectors, fused — beats either alone on almost every real corpus. If the ranking on this page looks more sensible than the one your system produced, you have found your answer.

The reasoning behind this

Related tools

Retrieval Debugger · Multigrid