Cost of Data Movement: Egress, Storage and Compute

Compute is the line item people plan for. The bytes moving between the stages that do the computing are charged at boundaries that do not appear in any architecture diagram, which is the only reason they surprise anyone.

Where bytes are charged

There are six kinds of charge and they behave differently enough that lumping them together makes the arithmetic useless.

Ingress is almost always free, which shapes the whole picture: getting data into a cloud costs nothing, getting it out costs the most, and that asymmetry is why the cheapest architecture is usually the one where the compute goes to the data rather than the other way round.

One boundary is invisible on any diagram and is worth checking for explicitly: traffic between services in the same region that leaves and re-enters the provider’s network because it is addressed by a public hostname. A managed database or object store reached through its public endpoint rather than a private one can be billed as internet egress in both directions, for traffic that never physically left the building. It is a DNS-level mistake with a per-gigabyte price, and the fix is a configuration change.

The cost model

For one full pipeline run over a corpus, with every price a parameter you fill in from your provider’s current rate card:

run_cost =
      D * S_raw   * p_ingest_requests        # fetch: per-object request cost
    + D * S_raw   * p_egress   * f_external  # only bytes that leave
    + D * S_text  * p_put                    # write extracted text
    + C * S_chunk * p_put                    # write chunks (C = chunks)
    + C * V * 4   * p_put                    # write vectors, 4 bytes/float
    + T * p_embed                            # the model call, per token

monthly_cost =
      (S_raw + S_text + S_chunk + S_vec) * D * p_storage_gb_month
    + Q * k * S_chunk * p_cross_zone         # query-time reads, per month

The term that catches people is the second-to-last on the first block — C * V * 4 — and the last line, because query-time transfer is proportional to traffic and therefore grows with success rather than with corpus size.

A worked example, assumptions labelled

All of the following are assumptions, not observations. Substitute yours.

• D = 200,000 documents, average raw size S_raw = 400 KB (PDF-heavy), extracted text S_text = 20 KB.
• C = 2.5M chunks of about 800 tokens, so T = 2×10⁹ tokens.
• Vector dimension V = 1,536 at float32.

Raw storage is 200,000 × 400 KB = 80 GB. Extracted text is 4 GB. Vectors are 2.5M × 1,536 × 4 bytes = 15.4 GB — before any index structure, which for HNSW adds the graph edges on top and can be a similar order again. So the vectors, which feel like the small derived thing, are four times the size of all the text they were derived from.

Now the movement. If extraction runs in the same region as the object store, the 80 GB never crosses a charged boundary. If it runs on a machine in another cloud, that is 80 GB of internet egress per full run, and a re-extraction pays it again. At an assumed egress rate of p per GB, moving the raw corpus once costs 80p while moving only the extracted text costs 4p — a factor of twenty for a decision about which machine runs the parser.

Request charges deserve a moment too. Writing 2.5M chunks as 2.5M individual objects is 2.5M PUTs; batching them into 10,000-chunk files is 250. At any published per-thousand-request rate that is a difference of four orders of magnitude on a line item most people never look at.

The levers, in order of size

• Co-locate compute with data. The largest single factor and it is a placement decision, not an optimisation. If the embedding provider, the object store and the vector store are in three different regions, every byte crosses two charged boundaries.
• Do not re-move what has not changed. Content hashing eliminates transfer as well as compute, and the 3%-change-rate arithmetic there applies unchanged to bytes.
• Move text, not originals. In the example above the extracted text is 5% of the raw bytes. Extract close to the source and ship the small thing.
• Batch small objects. Request charges are a function of object count, not size. Group chunks into files.
• Compress in transit and at rest. Text compresses roughly 3–4× with gzip and better with zstd; that is a direct multiplier on both storage and transfer for the largest text artefacts. Vectors do not compress usefully with a general-purpose compressor — for those the tool is quantisation.
• Lifecycle the raw bytes. Keep them, but in a colder tier after 90 days, remembering that a re-extraction then pays a retrieval charge. The comparison is storage-tier savings against the probability of a full re-extract.

Vectors are bigger than you think

The 15.4 GB above is the number worth carrying around, because it also sets your memory requirement: an in-memory index needs the vectors resident, plus the graph. Quantisation is the lever — scalar quantisation to int8 divides the vector bytes by four, and binary quantisation by thirty-two, at a cost in recall that is usually recovered by re-ranking a larger candidate set against the full vectors.

Dimension is the other lever, and it is free if the model supports truncation: halving V halves storage, memory and transfer for the vectors. The storage side of that trade is worth working through with your own numbers before committing to a dimension, because it is one of the few decisions here that is expensive to reverse.

Two habits keep this honest over time. Tag every resource with the pipeline stage that owns it, so the monthly bill can be read as a breakdown by stage rather than as a list of services — otherwise “object storage” is one number and nobody knows whether it is raw documents or vectors. And re-run the arithmetic above whenever the corpus doubles, because the terms scale differently: storage grows with the corpus, query-time transfer grows with traffic, and the term that dominates at ten thousand documents is rarely the one that dominates at a million.