Skip to content
Multigrid

RAG & retrieval

How to put your own documents in front of a model: chunking, ranking, filtering, evaluating and paying for the pipeline that does it.

Retrieval-augmented generation has a bad reputation among people who have shipped it, and the reason is that the interesting part is not the generation. A working system is mostly a search engine, and search engines fail in specific, boring, diagnosable ways: the chunk boundary fell in the wrong place, the filter was applied after the top-k instead of during it, the index has a document that was deleted last March.

These pages are written from that side. Where a page can show the code that does the thing, it shows the code. Where a claim comes from a published paper or a vendor’s own write-up, the source is named in the sentence that uses it — because the single most common defect in writing about retrieval is a confident number with no provenance.

RAG Explained: Retrieval-Augmented Generation From Scratch

A complete retrieval-augmented generation system in about sixty lines of Python, with no vector database and no framework, and what each part is actually for.

6 min read

Is RAG Dead Now That Context Windows Are Huge?

A cost model for stuffing the whole corpus into a long context versus retrieving from it, and the two situations where long context genuinely wins.

4 min read

Chunking for RAG: Size, Overlap and Semantic Splitting

How to pick a splitting strategy from the structure of your documents, what each strategy breaks, and what overlap costs you in index size.

5 min read

Hybrid Search: BM25 + Vectors, and Why You Need Both

The query classes each retrieval method fails on, and how reciprocal rank fusion combines two rankings without needing their scores to be comparable.

5 min read

Reranking: The Cheapest Accuracy Win in RAG

Why a cross-encoder ranks better than the retriever that fed it, what the published passage-ranking results show, and the latency budget it costs you.

5 min read

Query Rewriting and Multi-Query Retrieval

Turning one badly formed user question into several well-formed search queries, and merging what comes back without double-counting.

5 min read

HyDE: Retrieving With a Hypothetical Answer

Why embedding a made-up answer can retrieve better than embedding the real question, what the original paper claimed, and the failure it introduces.

5 min read

Metadata Filtering: The Part of RAG Everyone Skips

Pre-filter, post-filter and filtered graph traversal, why a selective filter quietly destroys recall, and the over-fetch arithmetic that tells you how bad it is.

5 min read

Evaluating a RAG System: Retrieval and Generation Separately

Why one end-to-end score cannot tell you which half is broken, the four-cell diagnostic that can, and how large an evaluation set has to be to see a change.

5 min read

Why Your RAG Returns the Right Chunk and the Wrong Answer

Seven ways a model mishandles context it was correctly given, how to tell them apart from the answer text, and what fixes each one.

5 min read

Citations and Source Attribution Users Can Verify

How to produce citations that point at a span of text rather than a document, and how to verify them programmatically before the user ever sees them.

5 min read

Graph RAG: When Relationships Matter More Than Similarity

The question types that top-k retrieval structurally cannot answer, what a graph index costs to build, and how to decide before you build one.

5 min read

Agentic RAG: Letting the Model Decide When to Search

Retrieval as a tool the model calls rather than a step that always runs, with a cost model for the trade and the failure modes the loop introduces.

5 min read

Keeping a RAG Index Fresh: Incremental Updates

Content-addressed chunk ids, the orphan problem when boundaries shift, and how to change embedding models without a maintenance window.

5 min read

Multi-Tenant RAG Without Leaking Between Customers

Namespaces versus filters versus separate indexes, and the four leak paths that are not the retrieval query at all.

5 min read

Contextual Retrieval: Adding Context to Chunks Before Embedding

Prepending an LLM-written summary of where a chunk sits in its document, the failure-rate reductions Anthropic published for it, and what the ingest pass costs.

5 min read

RAG Over Code: Why Text Chunking Fails on Repositories

What a fixed-window splitter destroys in source files, a working tree-sitter splitter that emits whole definitions, and why lexical search matters more here than anywhere else.

5 min read

Long-Context RAG: Stuffing 50 Chunks and What Breaks

What the published work says about where in a long context a model actually reads, and why raising k has a cost curve steeper than its recall curve.

5 min read

Caching in a RAG Pipeline: Three Layers Worth Having

Embedding, retrieval and answer caches, the invalidation bug that lives in each, and a hit-rate model for deciding whether any of them will pay.

5 min read

The Cost of RAG: A Full Per-Query Breakdown

A parameterised cost model for a retrieval pipeline, with every rate marked as an assumption to replace, and the sensitivity analysis that shows which line actually matters.

6 min read

Other topics

RAG & retrieval · Multigrid