Chunking Quality Inspector
Run five splitters over your own text and see the line, column and surrounding characters of every cut, plus what each one broke.
1 land inside a word, 0 inside a sentence. Median chunk is 800 characters (≈200 tokens at your ratio).
- Source length
- 893 characters
- Characters emitted across all chunks
- 993 (1.11× the source — the excess is overlap)
- Shortest / median / longest chunk
- 193 / 800 / 800
- Mean chunk
- 497 characters
- Estimated tokens per chunk at the median
- ≈200
| # | chars | ≈tokens | starts at | flags |
|---|---|---|---|---|
| 1 | 800 | 200 | 1:1 | mid-word |
| 2 | 193 | 48 | 25:48 | open code fence, very short |
Line:column is where each chunk starts in the source above.
- errorchunk 1 ends at line 30, column 62the cut lands in the middle of a word
Expected: a boundary at a whitespace character at least — a half word embeds as noise and matches nothing
…l embeds, and it still ret␣rieves. It simply⏎answers … - errorchunk 2contains an odd number of ``` markers, so a code block is split across this boundary
Expected: code fences kept whole — half a function retrieves as prose and the model reads it as prose
Everything on this page runs in your browser. Nothing you paste is uploaded, logged, or put in the URL — only the settings above the input are, so a configured tool can be linked to.
Why the boundary matters more than the size
Most chunking advice is about the number — 512, 800, 1000 — and the number matters far less than where the cuts land. A chunk that ends halfway through a sentence still embeds cleanly, still gets stored, still gets retrieved, and answers a slightly different question than the one its text appears to answer. Nothing downstream reports this. It shows up much later as a retriever that returns plausible chunks and a model that gives subtly wrong answers, which is the hardest failure in RAG to diagnose because every component looks healthy.
The overlap arithmetic people forget
Overlap is not free. At size 800 and overlap 400, every character is stored and embedded twice: your index doubles, your embedding bill doubles, and near-duplicate chunks start competing with each other for the same slots in the top-k. The ratio in the breakdown above is the honest number to check — if it says 1.6×, you are paying 60% more for the same corpus, and the question is whether the boundaries you rescued were worth it.
The structural flags are usually the highest-value finding here. A split code fence turns a function into two fragments that read as prose. An orphaned heading produces a chunk that consists of the words "## Refund policy" and nothing else — it will retrieve beautifully for refund questions and contain no answer. Both are invisible in any chunk-size histogram, and both are one line of splitter configuration to fix.