Storing Golden Files in Git Without Bloating the Repo

Most teams reach for Git LFS at the point where their golden corpus feels large, and most of them did not need to. The number worth computing first is not the size of the corpus but the size it adds per regeneration, and for text it is usually smaller than people fear.

Work out whether you actually have a problem

Do the arithmetic before choosing a tool. Take a normalised golden file for a model response — the reduced structure this cluster argues for, not raw prose — at roughly 1 KB. A corpus of 2,000 cases is then about 2 MB checked out. That is not a large repository by any measure; the working tree of a mid-sized web application dwarfs it.

Raw prose changes the input but not the conclusion. A 500-token answer stored verbatim is on the order of 2 KB, so the same 2,000 cases are about 4 MB. Both are fine. The figures here are stated assumptions, not measurements of any real repository, and the point of writing them down is that substituting your own numbers takes a minute and settles the question for your case rather than for a hypothetical one.

What is not fine, and what actually drives people to LFS, is a corpus containing binaries: recorded audio for a speech pipeline, rendered images, PDFs used as retrieval fixtures. A hundred WAV files at 2 MB each is 200 MB in one commit, and unlike text it does not compress or delta well.

Churn matters more than size

Git stores history, so the cost of a golden corpus is not its checked- out size but the sum of every version it has ever had. This is where regeneration discipline turns into a storage decision.

Suppose the 4 MB prose corpus above, regenerated in full fifty times over a year. Naively that is 200 MB of content. In practice it is much less: Git compresses objects and packs similar blobs against each other, and successive versions of the same answer differ in a few sentences, so the delta is small. Text is the case Git was designed for. The same fifty regenerations of a binary corpus store fifty complete copies, because a re-encoded audio file shares no useful structure with its predecessor.

That asymmetry is the whole decision. It also gives a second reason to snapshot reductions rather than prose: a reduced golden file changes in one field where a prose file changes in twenty lines, so the history costs less and — far more importantly — the diff stays reviewable.

Git LFS, and what it costs you

Git LFS replaces matching files with a small pointer in the commit and stores the content on a separate server. You declare the patterns with git lfs track, which writes them into .gitattributes, and that file must be committed for anyone else’s clone to behave the same way.

git lfs install
git lfs track "tests/fixtures/audio/**/*.wav"
git add .gitattributes
git commit -m "Track audio fixtures in LFS"

The trap, documented by the Git LFS project itself, is that tracking only affects files committed from that point on. Files already in history stay in history at full size, so a repository that is already large does not get smaller. Converting existing objects is a separate, history-rewriting operation:

git lfs migrate import --include="tests/fixtures/audio/**/*.wav" --everything

--everything covers all refs rather than the current branch, and git lfs migrate import --fixup converts files that your .gitattributes says should be in LFS but are not. Both rewrite commits, which means every collaborator re-clones. Plan it as an event, not as a Tuesday afternoon.

The cost that matters for golden files specifically is not storage quota. It is that an LFS-tracked file does not diff in a pull request: the reviewer sees a changed pointer hash. For an audio fixture that is irrelevant, because nobody was going to read the waveform. For a text golden file it destroys the entire purpose of having one, because the artefact exists to be read by a human deciding whether to approve it. Putting text goldens in LFS is the one clearly wrong answer here.

Object storage with a committed manifest

For genuinely large corpora — evaluation sets in the tens of thousands of cases, or anything with media in it — the arrangement that scales is to keep the data in object storage and commit a manifest: a small text file listing each fixture’s path and the SHA-256 of its content, plus the identifier of the dataset version.

This keeps the three properties you actually need from version control. The manifest diffs, so a review shows which fixtures changed and how many. The hashes pin content, so a test run can verify it fetched exactly what the commit expects and fail loudly rather than silently evaluating against different data. And the commit history records dataset changes as first-class events. The cost is a fetch step in CI and a cache to make it tolerable — the same caching problem as model weights, with the same solutions.

The rule that falls out of this

Keep in Git anything a human will read during review: normalised golden files, manifests, schemas, prompt versions. This is almost always the whole text corpus, and it is almost always smaller than people assume. Put in LFS anything binary that must be versioned alongside the code and that nobody reviews by eye. Put in object storage anything large enough that its history would dominate the repository, and commit a hashed manifest of it.

If your text corpus is genuinely too large for Git, the corpus is usually the problem rather than the storage. A corpus that no human reviews does not need to be a golden corpus at all; it needs to be a labelled dataset scored by machine-checkable assertions, which is a different artefact with different storage needs and a different failure mode.