What Happens to Prompt-Cached Content When You Leave a Vendor

The question behind this one is usually either “is my cached prompt content still sitting on their servers after I leave” or “can I take the cache with me”. The answers are: probably not for long, and definitely not — and the second answer has a cost attached that belongs in your migration estimate.

What is actually stored

Server-side prompt caching is not a store of your text. When a model processes a prompt, the expensive part is the prefill — running the whole prompt through the network to produce the attention keys and values for every token. A prompt cache keeps that intermediate compute state for a prefix of the prompt so that a later request with the same prefix can skip recomputing it. What lives in the cache is a tensor representation tied to a specific model, a specific serving configuration and usually a specific hardware layout.

Two consequences follow immediately. First, the entry is keyed by an exact prefix match, which is why cache guidance always tells you to put the stable material — system prompt, tool definitions, long documents — at the front, and the variable material at the back. Second, the entry is meaningless outside the model that produced it. There is no serialisation of it that another provider could load, because it encodes the internals of a different network.

The lifetime is short by design. Caches are sized to hold recent traffic, and entries expire on an idle timer measured in minutes for the common tier, with longer-lived options on some providers at a higher write cost. The mechanics and the economics of that trade-off are covered in prompt cache savings and prompt caching across providers; this page is only about what happens to it at the end of a relationship.

Why it is not an asset you can take

People ask about cache portability because the cache feels like something they paid to build. In cost terms that intuition is right: on providers that charge a premium for a cache write, you did pay to create the entry, and on providers that discount cached input, you have an ongoing benefit that will disappear. But the artefact itself is infrastructure, not data. The only durable thing is the prompt text you sent, and you already have that — it is in your repository, or it should be. If it is not, that is the actual finding of this page, and the fix is prompt versioning, not a contract clause.

The same is true of the reverse worry. A cache entry cannot be read back as text through the API — there is no endpoint that returns cache contents, and no field in a response that reveals another request’s cached prefix. What responses do expose is accounting: how many input tokens were served from cache on this request. Those counters are visible in the usage object, and their names differ by provider, which matters for your billing pipeline and is covered in what breaks in cost dashboards after a migration.

What account closure does to it

The honest general answer is that cache entries are among the shortest-lived things a provider holds about you, and by the time an account is closed there is realistically nothing left to delete — because the idle expiry has run many thousands of times over between your last request and the termination date. That is a mechanism argument, not a contractual one, and it is the reason cache is rarely named separately in a retention clause.

Where that reasoning does not hold, and where a security reviewer is right to push, is the boundary cases:

• The gap between last request and closure. If you stop sending traffic and close the account within minutes, entries may still exist during that window. In practice a migration takes weeks, so this only matters for a genuinely abrupt shutdown.
• Extended-lifetime cache tiers. Where a provider offers a longer-lived cache at a higher write price, the window in which entries survive after your last request is correspondingly longer. If you use one, know its documented lifetime.
• Cache is not the retention question. The prompt text that produced the cache entry is also request content, and request content is governed by the ordinary retention window — which is typically much longer than any cache lifetime. Asking about the cache when you meant to ask about content retention gets you a reassuring answer to the wrong question. The right question is in retention clauses and migration.

Which clause governs it

If you need a contractual answer rather than a mechanical one, the clause to read is almost never one that says “cache”. Cache entries are derived from customer content, so they normally fall under whichever definition your agreement uses for customer content or customer data, and they inherit that category’s deletion obligation. Two things worth confirming in your own agreement, in writing, if this is a procurement blocker:

• That the definition of customer content is broad enough to cover derived and intermediate representations, not only the literal request payload. Some definitions are written narrowly enough that a derived tensor is arguably outside them.
• That the deletion obligation applies to caches and other transient stores rather than only to the primary datastore, or alternatively that transient stores are covered by a stated maximum lifetime.

A zero-data-retention arrangement complicates this, because caching and zero retention are in tension by construction: one keeps derived state between requests, the other promises not to keep state. Providers resolve that tension in different ways and the resolution is a product-level fact rather than a universal one. If you operate under zero retention and rely on cache discounts in your cost model, confirm in writing that both apply to your account at once, because a cost model that assumes a discount you are not eligible for is wrong by exactly the discount rate.

What to plan for instead

The migration risk from prompt caching is not data loss. It is a cost cliff, and it is entirely predictable if you look for it.

On day one at a new provider, your cache hit rate is zero. It stays low for as long as it takes traffic to warm the new caches, and it may never return to the old rate, because cache economics differ: the minimum cacheable prefix length, whether the cache is engaged automatically or must be marked explicitly in the request, and the discount on a hit are all provider-specific choices. A prompt laid out to maximise hits under one provider’s rules is not automatically laid out well for another’s.

Three concrete things to do, in order:

1. Measure your current cached-token share before you move. Take the cached-input counter from your usage records over a representative week and divide by total input tokens. That single ratio is what you are about to lose and re-earn.
2. Model the ramp explicitly in the migration cost estimate: assume no cache benefit for the dual-run window and a partial benefit thereafter, rather than assuming the steady-state discount from day one. The arithmetic for this sits in estimating the true cost of a provider migration.
3. Re-check prompt layout against the new provider’s caching rules before the ramp, not after. Moving one variable field — a timestamp, a user id, a randomised example — out of the prefix is usually the difference between a cache that works and one that never hits, and it is a five-minute change if you find it early.

The summary a security reviewer wants: assume cache entries are transient, assume they are unrecoverable, and assume the retention question you actually care about is about request content, not about cache. The summary a finance reviewer wants: the discount does not migrate, and the gap belongs in the estimate.