Detecting When a Code Index Has Gone Stale

Search returns src/billing/legacy_charge.py, you open it, and the file is not there. Or it is there and the function is not, or the line numbers point at something else entirely. The index is behind the repository, and the fix is not to rebuild it until you know which of five things happened.

The symptom

Staleness shows up in four forms, roughly in order of how obvious they are:

• A result whose path does not exist. Opening it gives ENOENT or No such file or directory, and if your tool reads files to render snippets you will see that error in its logs rather than in the UI.
• A result whose path exists but whose quoted snippet does not appear in it. The chunk text was stored at index time and the file has since changed.
• A result with correct text but wrong line numbers, so the reader lands twenty lines off. This means the file changed above the chunk and the byte offsets were never updated.
• Recently added code is simply never returned. The hardest to notice, because there is no error — only an absence, usually reported as “search is bad” rather than as a bug.

Confirm it in one command

Before investigating, establish whether the index knows which commit it describes. If you stored a watermark, the check is immediate:

$ cat .git/code-index/head
3f9a1c2e8b7d4a6f0e5c9b2d1a8f7e6c5d4b3a29
$ git rev-parse HEAD
c81b47de9f0a2c3b5d6e7f8a9b0c1d2e3f4a5b60
$ git rev-list --count 3f9a1c2..HEAD
1,847

1,847 commits behind is a definitive answer and you can stop investigating the ranking. If the two hashes match and results are still stale, the watermark is lying — which is itself diagnostic, and points at cause 3 below.

If you did not store a watermark, get an approximate answer from the data you have: pick two hundred paths at random from the index and test each with git cat-file -e HEAD:path. The fraction that fail is your deletion drift. Anything above a percent or so means deletions are not being propagated at all.

Five causes, in order of likelihood

1. Deletions were never handled. By far the most common. The indexer walks the working tree and embeds what it finds, so a file that disappeared produces no event and its chunks live forever. Diagnostic: added and modified files are current, deleted ones persist. Fix: drive updates from git diff --name-status and act on D and the source side of R, as the commit-hook tutorial sets out.

2. The hook fires on too few events. A post-commit hook alone misses git pull, git merge when it fast-forwards, git checkout and git rebase. Diagnostic: locally authored changes are indexed, changes that arrived from other people are not — which produces the confusing report that search works for one developer and not another. Fix: install the same script behind post-merge, post-checkout and post-rewrite.

3. The watermark was written before the work finished. The pipeline records the new commit, then the embedding call fails, and nothing ever revisits that range because every subsequent diff starts after it. Diagnostic: a contiguous window of history is missing while both older and newer commits are present. Fix: write the watermark last, and to recover, diff from the last known-good commit and reprocess.

4. Silent partial failure. A batch of embeddings returned a rate-limit or a timeout, the error was caught and logged, and the run reported success. Diagnostic: files present in the file table with zero chunks in the vector store — a join that should return nothing and does not. Fix: make that join an assertion at the end of every run.

5. Branch mixing. The indexer ran against whatever was checked out, so the index contains one version of a file from main and another from a feature branch. Diagnostic: two chunks with the same path and overlapping line ranges but different text. Fix: index exactly one ref, or store the ref per chunk and filter.

The verification check

The check that settles all of the above compares content hashes rather than paths, because a path existing proves nothing about whether the bytes match.

# verify a sample of the index against the current commit
import subprocess, random

head = subprocess.check_output(["git","rev-parse","HEAD"], text=True).strip()
tracked = {}
for line in subprocess.check_output(["git","ls-tree","-r",head], text=True).splitlines():
    meta, path = line.split("\t")
    tracked[path] = meta.split()[2]        # the blob sha git already computed

sample = random.sample(index.all_files(), 500)
missing = [f.path for f in sample if f.path not in tracked]
drifted = [f.path for f in sample
           if f.path in tracked and f.blob_sha != tracked[f.path]]

print(f"indexed at {index.watermark()}  head {head}")
print(f"{len(missing)}/500 paths deleted since indexing")
print(f"{len(drifted)}/500 paths changed since indexing")

Two points make this cheap. git ls-tree -r gives you the blob sha of every file at a commit, already computed by git, so verification costs one git invocation rather than hashing the tree yourself. And storing that blob sha per file at index time costs forty bytes a row and is what makes the comparison possible at all — if the file table has only paths, add the column before anything else.

Sampling 500 files is enough to detect any drift above about half a percent, which is well below the level at which users notice. Run it against the full set only when the sample is dirty.

Making staleness visible

Staleness is a monitoring problem, not a search problem, and the three signals worth exporting are cheap. The commit distance between the watermark and the branch head is the primary one and it needs no new instrumentation. The age of the watermark in wall-clock time catches a pipeline that stopped running entirely. And the drifted fraction from the sampled check above, run hourly, catches the cases where the watermark is correct and the contents are not.

Show the first of these to the user. A search interface that says “indexed 4 commits ago” in small text next to the results converts a confusing wrong answer into an understandable one, and it is the single highest-value change you can make here — a stale result the reader knows is stale costs them ten seconds, while one they trust can cost an afternoon. The general treatment of that trade-off for document corpora is keeping a retrieval index fresh, and the structural fix on the write side is in the index architecture page.