Storing Vectors Cheaply: The Real Cost of 100M Embeddings

Vector storage cost is one of the few things in this field that is genuinely computable. The bytes are exact, the index overhead is a known function of one parameter, and the only input you have to look up is what a gigabyte of memory costs you this week.

The whole cost model in one formula

bytes = N * (d * bytes_per_dim  +  2 * M * 4) * replicas

  N             number of vectors
  d             dimensions
  bytes_per_dim 4 (float32), 2 (float16), 1 (int8), 0.125 (binary)
  M             HNSW links per node; 2*M*4 is the layer-0 link list
  replicas      1 for a single node, 2-3 for anything you page for

Everything else on your bill is a rounding error against this, with one exception noted at the end. The formula also makes the trade-offs legible: d and bytes_per_dim multiply, so truncating and quantising compound; M is additive and small, which is why tuning it to save memory is nearly always misdirected effort.

This page deliberately does not turn bytes into currency, because the per-gigabyte rate is the one input that goes stale and the arithmetic is the part that does not. Supply your own, and supply it in the right form: for a memory-resident index you are not really renting gigabytes, you are renting an instance, so the number that matters is which instance size the working set forces you into. Divide your total by the largest memory size you can get on one node to find the shard count, then multiply by replicas. A 627 GB index on 128 GB nodes is five shards; at two replicas, ten nodes. That is a much more useful answer than a per-gigabyte figure, because sharding also brings a coordinator, a rebalancing story and a failure mode per node.

Four scales, worked

At d = 1536, M = 16 (so 128 bytes of graph per vector) and a single replica:

               float32 vectors      graph        total      fits in
      1M          6.14 GB          0.13 GB       6.3 GB     a laptop
     10M         61.44 GB          1.28 GB      62.7 GB     one 128 GB box
    100M        614.40 GB         12.80 GB     627.2 GB     a 768 GB box, or 8 shards
      1B      6,144.00 GB        128.00 GB   6,272.0 GB     a cluster

same corpora, binary-quantised hot tier (0.125 bytes/dim) + float32 on SSD:
      1M          0.19 GB          0.13 GB       0.3 GB
     10M          1.92 GB          1.28 GB       3.2 GB
    100M         19.20 GB         12.80 GB      32.0 GB     one 64 GB box
      1B        192.00 GB        128.00 GB     320.0 GB     one large box

Read the 100M row twice. The same corpus is either a 768 GB machine or a 64 GB machine, purely as a function of one representation decision. That is a nearly 20× difference in the class of hardware you rent, and it is why quantisation with rescoring is the single highest-leverage thing on this page. Notice too that in the binary tier the HNSW graph is now 40% of the memory — the term you were right to ignore at float32 becomes the term worth tuning.

Replication multiplies the whole thing. Two replicas of the 100M float32 index is 1.25 TB of RAM; two replicas of the binary one is 64 GB. High availability is a decision you can afford at one representation and not at the other.

The one-off embedding bill

Generating the vectors is a capital cost, paid once per model generation. For 100 million chunks at 400 tokens, that is 40 billion input tokens:

40e9 tokens / 1e6 = 40,000 million-token units

at $0.02 per 1M   (OpenAI text-embedding-3-small, published Jan 2024)   $800
at $0.13 per 1M   (OpenAI text-embedding-3-large, published Jan 2024) $5,200

Set that against holding 627 GB of RAM for a year and the ranking is clear: the embedding call is a small one-time number and the memory is the recurring one. The practical consequence is that you should never choose a worse model to save on embedding cost at this scale — but you should absolutely choose a smaller dimension, because dimension is on the recurring side of the ledger.

Budget for paying it more than once, though. You will re-embed when you change models, when you change chunking, and when a fine-tune ships, and each of those is a full pass. Three passes over the corpus in the first eighteen months is an ordinary trajectory rather than a pessimistic one, so treat the single-pass figure as a unit cost and multiply by how often you expect to change your mind. The thing that reduces that multiplier is not a cheaper model; it is a gold query set good enough that most candidate changes get rejected before anyone starts a backfill.

Four levers, ranked

Deduplication belongs on the list as a special case of the third lever and is often the cheapest of all, because near-duplicates cost you memory and degrade retrieval at the same time. A corpus scraped from the web is routinely 20 to 40% redundant.

What the formula leaves out

• The text itself. 100 million chunks of 400 tokens is roughly 160 GB of raw text, or 40 to 50 GB compressed — less than the float32 vectors describing it. That inversion surprises people and it is a good sanity check that your storage plan is sane.
• Build headroom. An HNSW build wants the graph in memory plus working space. Provisioning exactly the steady-state figure gives you an index you cannot rebuild.
• Deleted-but-present vectors. Tombstoned rows occupy memory until compaction. A corpus with heavy churn can carry 20 or 30% dead weight between rebuilds; budget for it rather than discovering it.
• Metadata and payload. The filterable fields, their indexes and the ids. Small per row and not zero, and in a system with many indexed payload fields, not negligible either.
• Egress and snapshots. A 627 GB index copied for backup or moved between regions is a line item that appears in month two, not month one.