Skip to content
Multigrid

Embedding and searching code

Indexing a repository by meaning — chunk granularity, cross-language indexes, call graphs and keeping the index fresh.

Searching code by meaning is not searching prose with a different corpus. Code has a grammar, so the boundaries of a useful chunk are discoverable rather than guessed. It has explicit references, so the relationship between two files is a fact you can extract rather than a similarity you have to infer. And it changes in small, precisely described increments — a commit is a machine-readable statement of exactly what is now different — which means an index over a repository can be kept correct far more cheaply than an index over a document store.

These pages work outward from those three properties. Where an embedding is the right tool they say what it retrieves and what it misses; where a parser or a dependency graph does the job better they say so and show the extraction. Most of the difficulty in practice is not model choice but bookkeeping: knowing which chunk corresponds to which bytes of which commit, and noticing when that correspondence has quietly stopped being true.

Detecting Duplicated Code With Embeddings

Which kinds of duplication an embedding actually finds, how to pick a similarity threshold, and why the pair count is the real constraint.

9 min read

Designing a Repo-Scale Code Index That Updates Incrementally

The four stores a code index needs, why content-addressing the chunk is the load-bearing decision, and what full versus incremental re-indexing actually costs.

10 min read

Building a Code Search Engine From Cross-File Dependencies

How to extract an import and reference graph from a repository, and the three specific ways search ranking can use it.

10 min read

Re-Embedding a Codebase Incrementally on Every Commit

A git hook that re-embeds only the files a commit touched, with a content-hash cache and a recorded watermark so the index can prove it is current.

10 min read

Embedding Code Across Multiple Programming Languages in One Index

Why one embedding space ranks the same algorithm differently depending on the language it is written in, and the three mitigations that work.

10 min read

Function-Level and File-Level Code Embeddings: What Each One Retrieves

What a file-level vector actually encodes, why it drags irrelevant code into a result, and when the coarse granularity is the right answer anyway.

9 min read

Semantic Diff: Comparing Two Versions of a Function by Meaning, Not Text

Why a line-based diff overstates a rename and understates a logic change, and what tree-based and embedding-based comparison each recover.

9 min read

Finding Duplicated Business Logic Across a Large Codebase

Detecting the same rule reimplemented in different services and languages, and why the hard part is deciding which findings deserve a human.

10 min read

Choosing a Code Embedding Model for Search

How to build a query set from your own repository and measure retrieval precision, and which model properties actually change the index design.

10 min read

Parsing an Abstract Syntax Tree for Code Analysis

What an AST node actually holds, a walk over a short function shown node by node, and why error recovery decides whether a parser is usable on real repositories.

10 min read

What It Costs to Embed a Large Monorepo

A full derivation of the embedding bill for a 180,000-file repository, and why the vector store rather than the model is the recurring cost.

10 min read

Extracting a Call Graph From a Codebase

A runnable tree-sitter script that builds a caller-to-callee graph from a small Python project, and an honest account of what static extraction misses.

10 min read

Embedding Jupyter Notebooks for Search

How the .ipynb format is actually structured, why cell boundaries are the wrong chunk boundaries, and what to do with outputs.

9 min read

Detecting When a Code Index Has Gone Stale

Search returns a function that no longer exists: how to confirm the index is behind, find which of the five causes it is, and verify by commit hash.

9 min read

Other topics