Pinning a Dated OpenAI Model Snapshot Instead of an Alias

gpt-4o is not a model. It is a pointer to one, and OpenAI moves it. If your evaluation suite passed in March and fails in May with no change on your side, the pointer moving is the first thing to check and the easiest to rule out.

What an alias actually is

OpenAI ships two kinds of model id. A dated snapshot such as gpt-4o-2024-08-06 names one set of weights with one set of documented capabilities, and it does not change. An alias such as gpt-4o or gpt-4o-mini resolves to whichever snapshot OpenAI currently considers the best default for that name, and it is repointed when a new one ships — usually with a stabilisation period announced in the changelog, sometimes with less.

The alias is the right default for exploration and the wrong one for anything with a regression suite attached, for a reason that has nothing to do with quality: an alias makes your model a variable controlled by somebody else, deployed without your deploy. Whether the new snapshot is better is a separate question from whether you want it to arrive on a Tuesday afternoon.

One clarification, because the vocabulary borrowed from package management misleads. Pinning a snapshot is not like pinning a library version: there is no lockfile, no resolution step, and no warning when the alias moves. The model id is a string you send, and the only thing distinguishing a pinned request from an unpinned one is which string you sent. It costs nothing and requires no feature — but it is only available where the provider publishes a dated id, which is most models and not all. Knowing which of your model dependencies cannot be pinned at all is worth the five minutes it takes to check.

Step 1: find what you are running

Do not read this from the docs. Read it from a response, because the docs tell you what the alias resolves to for everyone and the response tells you what it resolved to for your organisation, on this request. Every Chat Completions response carries a model field with the resolved snapshot in it:

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "max_tokens": 1,
    "messages": [{"role": "user", "content": "hi"}]
  }' | jq '.model, .system_fingerprint'

The alias goes in, the snapshot comes out:

"gpt-4o-2024-08-06"
"fp_a7d06e42a7"

If you already log responses, you have this for your whole traffic history and can find the exact hour an alias moved by grouping requests by that field. That is a better answer than anything you will reconstruct afterwards.

Step 2: pin it

1. Take the snapshot id from step 1 verbatim. Do not retype it from memory; the date format is YYYY-MM-DD and a wrong day gives you model_not_found, not a nearest match.
2. Put it in configuration, not in a literal in your call site. An environment variable or a config row means the next re-pin is a deploy rather than a code review.
3. Send it. The request is otherwise identical — a snapshot id goes in the same field as an alias:

   import OpenAI from "openai";
   const client = new OpenAI();
   
   const MODEL = process.env.OPENAI_MODEL ?? "gpt-4o-2024-08-06";
   
   const res = await client.chat.completions.create({
     model: MODEL,
     messages: [
       { role: "system", content: "Reply with one sentence." },
       { role: "user", content: "Why is output priced above input?" },
     ],
   });
   
   console.log(res.model);              // gpt-4o-2024-08-06
   console.log(res.usage);              // token counts for this snapshot

4. Pin every model you call, not just the main one. The embedding model, the moderation model and the cheap classifier in the background job are all aliases too, and the classifier is the one nobody notices has drifted.

Step 3: verify the pin held

A pin that is silently ignored is worse than no pin, because you will stop checking. Assert it rather than trusting it. The cheapest version is one line in the code path that already handles the response:

if (res.model !== MODEL) {
  logger.warn({ requested: MODEL, served: res.model }, "model pin did not hold");
}

This is not paranoia about OpenAI. It catches the far more common case where something between you and the API — a proxy, a framework default, a stale environment variable in one replica — is rewriting the model field. In a fleet where one pod has the old config, the served model field is the only evidence you will get.

What breaks if you do not

• Capabilities differ between snapshots of one name. The clearest documented example is Structured Outputs: OpenAI introduced strict json_schema support with gpt-4o-2024-08-06, and the earlier gpt-4o-2024-05-13 snapshot does not have it. An application written against the alias while it pointed at the newer snapshot will 400 if it is ever run against the older one, and vice versa. Same name, different contract.
• Prompts regress without a code change. Instruction following, verbosity, refusal thresholds and formatting habits all shift between snapshots. Nothing about this is a bug; it is a different model. But it lands in your metrics as an unexplained change in output length or a jump in schema-validation failures, with a clean git log to look at.
• Cost and latency move. A new snapshot can be differently verbose and differently priced. If your unit economics are computed from tokens per request, the denominator changed under you.
• Reproducibility is gone. The seed parameter and system fingerprint only mean anything relative to a fixed model. A seeded request against an alias is not a reproducible request.
• Token accounting can shift. A snapshot that moves to a different tokenizer changes what your prompt costs without changing a character of it. That has already happened once across the GPT-4 to GPT-4o boundary — cl100k_base against o200k_base — and any local token estimate keyed to the old encoding silently becomes an estimate of a different quantity.

What pinning does not protect you from is worth stating too, so that the pin is not asked to carry more than it can. Serving infrastructure changes underneath a fixed snapshot — different hardware, different batching, different numerical kernels — and that is enough to make two identical requests to one pinned model return different text. A pin fixes the weights and the documented capabilities. It does not make inference deterministic, which is why system_fingerprint exists as a separate signal and why reproducibility is best-effort even with a seed.

Step 4: schedule the re-pin

Pinning is not a decision to stop upgrading, it is a decision to upgrade deliberately, and a pin nobody revisits turns into the thing that gets shut down under you. This page is marked refresh for the same reason your pin should be.

1. Put the deprecations page on a quarterly review — deprecated and shut down are different dates and only the second one is an outage.
2. When a new snapshot appears, run your evaluation set against both ids in parallel. You cannot do this at all without a pin, which is the underrated benefit: two fixed ids are comparable, an alias and a fixed id are not.
3. Move the config value, watch the same metrics you watch for a normal deploy, and keep the old id available for one rollback window.

Two failure modes to design against while you are here. A pin that has been in place long enough to be forgotten is the one that gets shut down under you, so the review has to be a calendar item rather than an intention. And a pin held because the migration is frightening rather than because the evaluation failed is technical debt with an externally imposed deadline: the shutdown date arrives whether or not the work is done, which is the worst kind of deadline to discover late. If you find yourself unable to say what would have to be true to move, that is the signal to run the comparison now rather than at the announcement.