Embedding Backfill Time Estimator

Turns corpus size, batch size and your rate limits into the wall-clock time an embedding backfill takes, and names the limit that binds first.

Items to embedChunks, not documents, if you chunk first.

Tokens per item

Items per requestBatch size the embedding endpoint accepts.

Requests-per-minute limitYour account's RPM ceiling.

RPM

Tokens-per-minute limitYour account's TPM ceiling.

TPM

Round trip per requestWall-clock latency of one batched call.

Concurrent workersHow many requests you keep in flight.

Retry and failure factor1.05 = 5% of the work done twice. Backfills always redo some.

Embedding rate per 1M tokensExample figure — replace it with the rate on your own bill.

$ / 1M

Wall clock to finish the backfill

3.1 h

The tokens per minute limit binds first, at 30 requests per minute. Raising anything else changes nothing until you raise that.

Total tokens to embed: 175,000,000
Requests: 5,209
Tokens per request: 33,600
Ceiling from the RPM limit: 3,000 req/min
Ceiling from the TPM limit: 30 req/min
Ceiling from your concurrency: 320 req/min
Binding limit: tokens per minute
Effective rate: 47.6 items/s
Cost at $0.10 per 1M tokens: $18.38

What this assumes: the three ceilings are independent and the slowest one governs, which is true of a token bucket that refills continuously and false in the first seconds of a burst. Retries are billed — a failed request that returned a 429 before any work usually is not, so treat the retry factor as covering timeouts and 5xx rather than rate-limit rejections. Every item is assumed to be the same length; a corpus with a long tail of huge documents hits the TPM ceiling earlier than this says.

Which limit binds, and why it is rarely the one you expect

A backfill is throttled by whichever of three ceilings is lowest, and two of them are outside your control. The requests-per-minute limit caps how many calls you may make. The tokens-per-minute limit caps how much text those calls may carry. Your own concurrency — workers divided by round-trip latency — caps how many you can actually keep in flight. They are quoted in different units, so people compare them by instinct and get it wrong.

Batch size is what moves the answer, because it converts one ceiling into the other. Every item you add to a request buys RPM headroom and spends TPM headroom: requests fall by the batch factor while tokens per minute stay flat. There is a batch size at which the two ceilings cross, and that is the fastest you can go. Below it you are wasting request slots; above it you are queueing behind the token bucket and the extra batching buys nothing. Change the batch field here and watch which row in the breakdown becomes the smallest — the crossing point is visible in a few keystrokes.

What this does not model: a provider that meters tokens per minute over a sliding window rather than a bucket, per-organisation limits shared with your production traffic (a backfill running flat out will starve it — run at a fraction of the ceiling if it shares a key), or the write side, where a vector store that ingests slower than you embed becomes the real bottleneck.

Embedding Backfill Time Estimator

Which limit binds, and why it is rarely the one you expect

The reasoning behind this

Related tools