Keying an Embedding Cache So a Model Migration Can't Poison It

A cache key is a claim: given this key, the value is what the function would have returned. For an embedding cache that claim only holds if the key names every input the function takes — and the text is only one of them.

The key identifies a function, not a string

Write down the call you are caching in full, with every argument visible:

embed(
    text,
    model="acme-embed-3",
    dimensions=1024,
    input_type="document",
    truncate="end",
)  ->  list[float]

The cache key must be a function of everything on the left. Hashing only text asserts that the other four arguments are constants of the universe. They are configuration, which means they change, and the day one of them changes the cache starts returning answers to a question nobody asked. The failure that follows is described in migrating an embedding cache when you change models; this page is how you never have it.

The rule generalises past this one cache: any memoisation of a remote call has to key on the whole request, and the parts people leave out are the ones that live in environment variables rather than in the function signature.

What has to be in it

• Provider. Two vendors can ship a model with a similar name, and a self-hosted deployment of an open-weights model is a third case again. Cheap to include, and it makes the key readable.
• Model identifier, exactly as sent on the wire. Not a friendly alias from your config. If you send an alias that the provider resolves to a dated snapshot, that alias is a moving target — the same key can map to two different functions across a provider-side update. Where the provider exposes pinned snapshot names, send and key on the pinned name.
• Output dimension. Where a model supports a requested output width, each width is a distinct function. Two vectors of different lengths for the same text are not the same value, and a truncated vector is not a prefix-compatible substitute even when the model is trained to allow truncation.
• Input type or task type. Several embedding APIs take a parameter distinguishing a query from a document, and asymmetric models produce genuinely different vectors for the same string under each. If that parameter is not in the key, a string that appears both as a document and as a query returns whichever was cached first. This is the second most common poisoning after the model itself and it does not need a migration to happen — it happens on a Tuesday.
• Truncation and other request-shaping parameters.Anything that decides what the provider actually sees.
• A preprocessing version of your own. If you lowercase, collapse whitespace, strip markup, prepend an instruction prefix, or template the chunk before sending it, then your preprocessing is part of the function. Bump an integer when you change it. This is the component people omit and then spend a day confused by, because the change is in their own code rather than the provider’s.
• A hash of the exact bytes sent. SHA-256 over the UTF-8 encoding of the post-preprocessing text. Truncating the digest to 128 bits is fine; the collision probability at any corpus size you will hold is not the risk in this system.

Why the order of the components matters

It is tempting to hash the whole tuple into one opaque digest. Do not. Concatenate the identity components as a readable prefix and put the text digest last.

acme|acme-embed-3-2026-02|1024|document|pp7:9f2c1e...

The prefix buys three things that an opaque hash cannot.

• Enumeration. Every entry for one model shares a prefix, so you can scan them, count them, and delete them as a group — a SCAN with a match pattern in Redis, a key prefix in an object store, a LIKE in a database table. Without it, the only way to remove one model’s entries is to flush everything.
• Debuggability. A key you can read tells you which function produced the value. When somebody reports a strange result you can look at the key in the log and know the model, the dimension and the input type without a lookup table.
• Isolation by construction. Two models cannot collide because their keys differ in the first field, not merely with high probability in a digest. That is a property you can state, not one you have to trust.

Use a separator that cannot appear in any component. Model identifiers contain hyphens, dots and often slashes for self-hosted repositories, so a pipe or a NUL-adjacent character is safer than a dash. And keep the field count fixed — if a component is not applicable, write an explicit - rather than omitting it, so that a five-field key and a four-field key can never be the same string.

Store the components next to the value

The key tells you what a value is only as long as you have the key. Put the same identity in the stored value as well, as a small envelope around the vector.

{
  "v": [0.0123, -0.0456, ...],
  "model": "acme-embed-3-2026-02",
  "dim": 1024,
  "input_type": "document",
  "preproc": 7,
  "created_at": "2026-08-11T09:31:00Z"
}

The envelope costs a few dozen bytes against a vector of several kilobytes, and it turns two otherwise-impossible things into ordinary queries. First, a consistency assertion: on read, compare the envelope’s model against the model you expect and raise if they differ. That converts any future keying bug from a silent quality regression into an exception with both values in the message. Second, an audit: you can answer “how much of this cache is still the old model” without re-deriving anything.

Write the same fields into the vector store record too, not just the cache. That is what makes the completeness check in auditing whether a re-embedding actually finished possible at all.

Running two models through one cache

With identity in the key, two models coexist with no further work — the keyspaces are disjoint. That is what makes a staged migration possible rather than a big-bang swap.

During the migration you will want to warm the new model’s entries before you depend on them. Read the old model’s keys by prefix, recover the source text — which means you must have stored it, or be able to fetch it by identifier from the system of record — and write new-model entries under the new prefix. Nothing about this touches the entries currently serving traffic, so it can run at whatever rate your provider rate limits allow, over days, with no risk to the live path.

The one thing to decide deliberately is when the old prefix goes. Keep it until you would no longer roll back; delete it by prefix scan when you would not. Do not attach a short TTL to embedding cache entries as a substitute for that decision — a TTL is an unreviewed rolling flush, and it will expire your old entries during exactly the week you want them.

Building it

1. Move every embedding call in the codebase behind one function. If two call sites can construct a key, they will eventually construct different keys.
2. Make the model configuration a single object holding provider, model id, dimension, input type and preprocessing version, and pass it explicitly rather than reading environment variables inside the key builder. A key builder that reads global state cannot be tested.
3. Write the key builder to raise if any component is empty. A blank model identifier produced by a missing environment variable would otherwise silently create a shared keyspace across environments.
4. Store the envelope, and assert on read that the envelope’s model matches the requested model. Count the mismatches as a metric rather than only raising, so a partial rollout is visible.
5. Add a test that changes each component in turn and asserts the key changes. This is five lines and it is the test that would have caught every incident on this page.
6. Add a second test that asserts the key is stable across processes and machines — no hash() with a randomised seed, no dictionary iteration order, no locale-dependent case folding. A key that differs between two workers is a cache with a 0% hit rate that reports no errors at all.