Estimating the True Cost of a Provider Migration

The number that starts a migration conversation is a price ratio: the new provider is some fraction of the old one per million tokens. The number that decides whether the migration was worth doing is a payback period, and it is dominated by items that are not per-token at all.

Why the per-token comparison is not the answer

Three things break the direct comparison, and all three are mechanical rather than a matter of opinion.

• Tokenisers differ, so the same text is a different number of tokens. Providers price per token, and two providers’ tokenisers segment the same string differently — different vocabulary size, different merge rules, different handling of whitespace and of non-Latin scripts. A price that is 20% lower per token on a corpus that tokenises 15% longer is a 8% saving, not a 20% one. The mechanism is in tokenizer comparison and the effect is largest on non-English text, covered in the tokenizer language tax.
• Output length is a model property, not a constant. If the new model is more verbose for the same prompt, you pay the difference at the output rate, which is normally the dearer of the two. This is measurable on your own replay corpus before you commit and is one of the most commonly skipped checks.
• Discounts do not transfer. Cache hit rate, batch-tier pricing and any committed-spend discount are all per-relationship. Your effective rate at the old provider is not its list price, and comparing the new list price against your old effective rate flatters the move.

So the first thing to compute is not a saving. It is an effective blended rate on each side, derived from your own traffic mix rather than from a rate card.

The line items

Everything below is a one-off unless marked recurring. Symbols are used so you can substitute your own values; nothing on this page asserts a price.

Engineering

The adapter work, the test updates, the deploy plumbing, the two weeks of small surprises afterwards. Estimate it as E_days × D, where D is your fully loaded cost per engineer-day. The systematic error here is forgetting the second engineer: a migration that one person builds usually needs a second person for review, on-call cover during the ramp, and the dashboards.

Re-embedding

If you are also changing embedding provider, every vector must be rebuilt, because embeddings from different models are not comparable — this is not a preference, the spaces are unrelated. Cost is N_chunks × T_chunk × P_embed, plus the write cost and the index rebuild in your vector store, plus a period of dual indexes. Note that this line is zero if you keep the embedding provider, which is a strong argument for treating the two migrations as separate projects.

A frequently missed sub-item: a re-embedding changes retrieval results, so your retrieval evaluation must be re-run and your similarity thresholds re-tuned. See testing that a retriever is stable after a reindex.

Dual-run overlap (recurring, bounded)

During the ramp you pay both providers. If W is the overlap in months and d is the fraction of traffic duplicated — sending the same request to both for comparison rather than splitting traffic — the overlap cost is roughly W × M × (1 + d) against a normal monthly spend M, minus whatever the split saves at the cheaper rate. Note that d = 0 for a percentage split and d = 1 for full shadow traffic, and the choice between them is a real decision: shadow traffic gives you a like-for-like quality comparison and doubles the bill.

Evaluation re-runs

Every model change invalidates your baselines, so the eval suite runs repeatedly through the migration rather than once. Cost is R × C_eval where R is the number of full sweeps and C_eval is the cost of one. Teams reliably underestimate R: expect one before, one per candidate model, one per prompt revision made to fix a regression, and one after the ramp.

Stranded commitment

Unused committed spend at the old provider, from the notice period and any minimum commitment. This is sunk rather than avoidable, and it belongs in the estimate as a one-off so that the recurring comparison is not polluted by it.

Cache warm-up

The discount you had and temporarily do not, described in what happens to prompt-cached content on exit. Model it as the cached-token share of your old bill, applied for the overlap period and half of it for a month after.

A worked example on labelled assumptions

Every number below is an assumption you replace. They are chosen to be round, not to be typical, and no figure here is a claim about any provider’s prices.

ASSUMPTIONS (substitute your own)
  M         monthly LLM spend today          = 20,000 currency units
  r         new provider effective rate       = 0.70 x old
              (after tokenizer + verbosity adjustment, not list price)
  E_days    engineering effort                = 25 engineer-days
  D         loaded cost per engineer-day      = 700
  N_chunks  chunks to re-embed                = 0   (embeddings unchanged)
  W         dual-run overlap                  = 1.5 months
  d         duplicated traffic fraction       = 0.5 (half shadowed)
  R         full eval sweeps                  = 6
  C_eval    cost of one eval sweep            = 400
  K         stranded commitment               = 5,000
  c         cached-token share of old bill    = 0.15

ONE-OFF COSTS
  engineering        E_days * D          = 25 * 700      = 17,500
  re-embedding       N * T * P_embed                     =      0
  eval re-runs       R * C_eval          = 6 * 400       =  2,400
  stranded commit    K                                   =  5,000
  dual-run extra     W * M * d           = 1.5*20000*0.5 = 15,000
  cache warm-up      1.5 * c * M         = 1.5*0.15*20000=  4,500
                                          ------------------------
  TOTAL ONE-OFF                                          = 44,400

RECURRING SAVING
  monthly saving     M * (1 - r)         = 20000 * 0.30  =  6,000 / month

The shape of that table is the deliverable, not the totals. Two things in it are worth arguing with before you copy it: the dual-run extra is the second largest line and is entirely a function of a decision you control (d), and the cache warm-up line is invisible in most estimates despite being comparable to the eval budget.

Payback and break-even

With one-off cost C and monthly saving S, the payback period is simply C / S. On the assumptions above:

payback = C / S = 44,400 / 6,000 = 7.4 months

The useful inversion is the break-even rate ratio: given your one-off cost and the horizon H over which you are willing to pay it back, how much cheaper must the new provider actually be?

break-even r  =  1 - C / (M * H)

with C = 44,400, M = 20,000, H = 12 months:
  r = 1 - 44,400 / 240,000 = 0.815

i.e. the new provider must be at least ~18.5% cheaper on an
effective basis to pay back inside a year on these assumptions.

Run that inversion first, before any technical work. It converts an open-ended question — is this worth doing — into a threshold test you can settle with a tokeniser count and a replay of a few hundred real requests. A migration whose break-even ratio is 0.95 is not a cost migration; if you are doing it anyway, be honest that the reason is capability, reliability or contract terms rather than price.

Which input the answer is most sensitive to

Vary one input at a time from the worked example and the ordering is consistent enough to be worth internalising.

• The effective rate ratio r dominates everything, because it is the only input that is recurring. Moving r from 0.70 to 0.80 cuts the monthly saving by a third and pushes payback from 7.4 months to 11.1. This is why the tokeniser and verbosity adjustments matter so much: they move r, and every other line is a one-off.
• The dual-run duplication fraction d is the largest controllable one-off. Halving d from 0.5 to 0.25 in the example removes 7,500 and brings payback to 6.2 months. Shadow traffic is worth buying — it is the only way to compare two providers on identical inputs — but buy it for a sample of traffic and a bounded window, not for everything for the whole ramp.
• Engineering effort is the input most often wrong, and it is wrong in one direction. The estimate that omits the second engineer, the observability rework and the two fix-up weeks is not slightly low, it is low by a factor. The concrete rework is described in migrating observability fields and what breaks in cost dashboards.
• Re-embedding is either zero or enormous. There is no middle. If it is non-zero, it usually exceeds every other line combined for a corpus of any size, which is the argument for not changing embedding and generation providers in the same quarter.

Finally, a scope note on what this model does not price: risk. A migration carries a probability of an incident, and an incident has a cost that is real but not derivable from a rate card. The defence against it is procedural rather than financial — a ramp you can reverse and a rollback trigger agreed in advance — and its cost is already inside the dual-run and engineering lines.