What It Costs to Embed a Large Monorepo

The one-off cost of embedding a large monorepo is almost always smaller than people budget for, and the recurring cost of holding the vectors is almost always larger. The arithmetic below is what makes that comparison, and every input is stated so you can substitute your own.

The inputs, all of them assumptions

Nothing here is measured on a real repository. These are stated assumptions, and the arithmetic is the part worth keeping — replace each figure with one from your own git ls-files output and a tokeniser run and the method still holds.

• 180,000 indexable source files after excluding vendored, generated and binary paths. That is a large monorepo.
• 3.2 KB mean file size, which is roughly 90 lines of typical source.
• 3.6 bytes per token for source code. Code tokenises less efficiently than English prose because identifiers split and punctuation is dense; measure yours rather than trusting this one.
• 15% overlap overhead from chunking, since consecutive chunks share context.
• $0.02 per million tokens, OpenAI’s listed price for text-embedding-3-small, and $0.13 per million for text-embedding-3-large, on its published pricing page as read on 11 August 2026.
• 1,536 dimensions, float32 for the storage calculation, and 400 tokens as the mean chunk size.

From bytes to tokens

180,000 files x 3,200 bytes            =    576,000,000 bytes
576,000,000 bytes / 3.6 bytes-per-token =   160,000,000 tokens
160,000,000 x 1.15 (overlap)            =   184,000,000 tokens billed

The bytes-per-token figure is the input most worth replacing with a measurement, and it is easy to get: run your tokeniser over a random thousand files and divide total bytes by total tokens. Minified JavaScript, deeply nested JSON fixtures and non-Latin identifiers all push the ratio down — meaning more tokens per byte and a larger bill — and heavily commented English-language code pushes it up. A ratio of 2.5 rather than 3.6 would raise the token count by 44%, so this single number carries most of the uncertainty in the result.

The 15% overlap figure deserves the same scrutiny. It assumes chunks split on syntactic boundaries with a small amount of shared context — an import block or a class signature repeated into each method chunk. If instead you prepend a file summary and a list of referenced symbols to every chunk, as several designs do to improve retrieval, the overhead is not 15% but can approach 50%, because the added context is a fixed cost paid once per chunk rather than once per file. That choice is made for retrieval-quality reasons and it is usually right; it simply needs to be counted here rather than discovered on the invoice.

The embedding bill

text-embedding-3-small
  184 million tokens x $0.02 / 1,000,000  =  $3.68

text-embedding-3-large
  184 million tokens x $0.13 / 1,000,000  =  $23.92

Under four dollars to embed every source file in a 180,000-file monorepo with the cheaper model. That is the number worth carrying away, because it changes how you should think about the whole project: the embedding call is not a cost to optimise. It is not worth building an elaborate incremental system to avoid $3.68 of compute, and it is not worth choosing the cheaper model to save twenty dollars once. Choose the model that retrieves better and re-embed whenever you want to.

Two caveats keep this honest. Rate limits, not price, are what make a first full index take hours rather than minutes — a provider’s tokens-per-minute ceiling divides into 184 million to give the wall clock, and that is the number to check before scheduling the run. And a self-hosted embedding model has no per-token price at all but does have a GPU-hour price; at these volumes the hosted option is usually cheaper than the engineering time to set up the alternative, and the reason to self-host is that source code is leaving your network, not money.

Storage is the larger number

chunks     = 184,000,000 tokens / 400 tokens-per-chunk = 460,000
per vector = 1,536 dims x 4 bytes (float32)           = 6,144 bytes
raw        = 460,000 x 6,144                          = 2.83 GB

ANN graph overhead, HNSW at M=16
  ~16 x 2 neighbours x 4 bytes = 128 bytes/node
  460,000 x 128                                       = 0.06 GB

resident total                                        ~ 2.9 GB

That 2.9 GB has to be in memory for a graph index to perform, which means an instance sized above it, running continuously. Whatever that instance costs per month, it costs it every month, against a one-off $3.68 — so the ratio between the two is not close, and it is the storage side that deserves the engineering attention. What vector storage actually costs works through the pricing shapes.

Two levers move it substantially. Truncating a Matryoshka-trained embedding from 1,536 to 512 dimensions cuts the vector bytes to 2,048 and the resident total to roughly 0.94 GB, for an accuracy loss you can measure on a hundred-query set before committing. Scalar quantisation to int8 divides the raw figure by four independently of that. Both are cheap to evaluate and either can pay for itself many times over the life of the index — what dimension buys you is the general treatment.

What the storage figure does not include is the working memory the index needs while it is being built. Constructing an HNSW graph over 460,000 vectors holds the vectors plus the partially built graph plus the insertion buffers, and peak usage during a build commonly runs to something like twice the resident steady-state size. A machine sized for 2.9 GB will fail during the first full index and succeed on every incremental update afterwards, which is a confusing way to discover the constraint. Size for the build, not for the steady state, or build on a larger machine once and load the result.

Steady state after the first run

assume 900 changed files per day (0.5% of the repo)

900 x 3,200 bytes / 3.6 x 1.15  =    920,000 tokens/day
920,000 x $0.02 / 1,000,000     =        $0.018 / day
                                =        $0.55 / month

Fifty-five cents a month of embedding compute against a gigabyte-scale index held in RAM continuously. The conclusion is the same one the architecture page reaches from the other direction: the reason to build incremental re-embedding is latency and freshness, not cost. If you find yourself justifying an incremental pipeline on the embedding bill, the justification is wrong even though the pipeline is right.

One cost this derivation omits entirely, and should not be assumed away: query-time embedding. Every search embeds the query, so a tool used by 300 developers making 40 searches a day at 20 tokens each is 240,000 tokens a day — trivial in money, but it is a synchronous call on the latency path of every search, and that is a real design constraint even though it is not a real line on the bill. Cache repeated queries and the constraint largely goes away.