The Hidden Costs: Embeddings, Storage and Egress

6 min read · updated August 3, 2026

Inference has a price list, which is why it gets modelled. The other costs of running an AI feature arrive on four different invoices, none of which says “AI” on it, and they are the reason a carefully forecast bill still comes in high.

Why these go missing

Three structural reasons, and knowing them tells you where to look. They land on someone else’s budget — the vector database is on the infrastructure invoice, the log ingestion is on the observability contract. They are small per unit and large in aggregate, so they fail the “is this worth modelling” test individually and pass it collectively. And several of them scale with a variable nobody is watching: corpus size, retention period, or the number of evaluation runs, none of which appear in a per-request cost model.

The practical consequence is that nobody owns the total. The engineer who knows the inference bill does not see the storage line; the platform team who sees the storage line does not know it is caused by an AI feature. So the first move is not an optimisation at all — it is to pull four invoices for the same month and attribute each line to a feature. That exercise takes an afternoon and it is the only way the number below becomes real rather than borrowed.

The inventory

Line	Description
embedding (ingest)	One pass over the corpus at an embedding model's per-token price. Usually the cheapest line here and the one people most expect to be expensive.
embedding (re-index)	The same cost again, every time you change embedding model, chunking strategy or preprocessing. This is a project cost that recurs unpredictably.
vector storage	Per GB-month, driven by vector count times dimensions times bytes per element, plus index overhead. Grows with the corpus, not with traffic.
vector query	Often a separate per-query or per-read-unit charge. Scales with traffic and with top-k.
log and trace storage	Prompts and responses are large and are kept for debugging, evaluation and compliance. Ingest-priced observability platforms make this the surprise line.
egress	Per GB out of a cloud, and per GB across regions. Matters when the model, your app and your data store are not in the same place.
evaluation and development	Every eval run, every CI check, every engineer iterating on a prompt is inference you pay for and that no customer generated.
moderation and safety	A classifier call on the way in, another on the way out. Cheap each, applied to everything.
human review	Any share of output checked by a person. Almost always the largest line on this table when it exists at all.

Sizing each line

Embeddings

ingest_cost = corpus_tokens * P_embed / 1e6

50M tokens at an assumed $0.02 per million = $1.00

A dollar. Embeddings are almost never the problem, and it is worth computing once so that the fear of them stops influencing architecture. What is expensive is what the embeddings imply.

Vector storage

raw_bytes = n_vectors * dims * bytes_per_element
stored    = raw_bytes * index_overhead     (roughly 1.5x - 2x)

5,000,000 chunks * 1,536 dims * 4 bytes = 30.7 GB
at 2x overhead                          = ~61 GB
at an assumed $0.25 / GB-month          = ~$15 / month

Small here, and it scales linearly with the corpus while your revenue may not. Two levers worth knowing before it is large: quantising to 8-bit or 4-bit elements cuts the first term by 4× or 8× with modest recall loss, and a smaller embedding dimension cuts it proportionally. Both are far easier to choose at design time than to migrate to later.

Logs

bytes_per_request ~= (T_in + T_out) * 4     bytes per token, roughly
monthly_GB        = requests * bytes_per_request / 1e9

200,000 requests at 3,000 tokens = 2.4 GB / month  (object storage: pennies)
                                    but on a $2/GB ingest platform: ~$5/month

At 10M requests / month:  120 GB  ->  $240 / month on the same platform,
and it accumulates if retention is longer than a month.

The number that matters here is the price per GB of wherever the logs land, and it varies by two orders of magnitude between object storage and an ingest-priced observability product. Sampling full bodies — keep 1% with full text, keep metadata for all — is the standard fix and typically costs nothing you actually needed.

Evaluation

eval_cost = cases * models * runs_per_month * cost_per_case

400 cases * 3 models * 40 runs * $0.01 = $480 / month

Every pull request that triggers the suite is an eval run. This line grows with engineering activity rather than with traffic, which means it is largest exactly when the product is smallest — and it is the one line here you should be reluctant to cut, since it is what stops the expensive mistakes.

Human review

review_cost = requests * review_rate * minutes_each / 60 * hourly

200,000 * 2% * 0.5 min / 60 * $25 = 4,000 * 0.00833 * 25 = $833 / month

Reviewing 2% of output for thirty seconds each costs about as much as a substantial share of the inference that produced it. If your quality strategy involves a human in the loop, that decision is a cost-of-goods decision and belongs in the margin model, not in a quality document.

Computing your own share

There is no correct percentage. The share is whatever your inventory says, and it varies enormously between a chat product with no retrieval and a document platform with a review queue. So compute it:

non_inference_share = other / (inference + other)

Assembled from the assumed figures above, for a product
spending $8,000 / month on inference:

  embeddings, amortised .....    $50
  vector storage + query ....   $180
  logs and traces ...........   $240
  egress ....................    $90
  evaluation and dev ........   $480
  moderation ................   $120
  human review ..............   $850
                       other = $2,010

share = 2,010 / (8,000 + 2,010) = 20.1%

Twenty percent for that particular set of assumptions. Change one — a product with no human review drops to 13%; one on an ingest-priced log platform at ten times the volume goes well past 30%. Run your own inventory rather than borrowing this one; it is an afternoon with four invoices and it is the only version of the number that is true.

The ones that grow on their own

Most of these lines are proportional to traffic and therefore behave predictably in a forecast. Three are not, and they are the ones that cause surprises:

Storage accumulates. Logs and vectors are stocks, not flows. A constant request rate produces a monotonically rising storage bill unless there is a retention policy, and “we will add retention later” is how a small line becomes a large one. Set the policy when you create the bucket.
Corpus growth is a customer variable. Vector storage scales with what your customers upload, which is unrelated to what they pay unless you made it related. An enterprise customer with a large archive can cost more in storage than in inference.
Re-indexing is a step function. Changing embedding model means re-embedding everything and rebuilding the index — a cost that is zero for a year and then substantial in one week. Budget for it as an occasional project, and note that its size grows with the corpus, so the longer you defer it the more it costs.

A last note on egress, because it is the one people find hardest to believe. Moving data out of a cloud is priced per gigabyte and moving it between regions often is too. If your application runs in one cloud and your model in another, every request crosses that boundary twice. It is small per request and worth checking once, because co-locating is usually free to do and impossible to retrofit cheaply.