Skip to content
Multigrid

Pinning a Phi Checkpoint on Hugging Face

9 min read · updated August 11, 2026

from_pretrained("microsoft/phi-4") resolves to whatever is on main at the moment it runs. That is a moving dependency in the middle of your inference stack, and pinning it is four lines of change.

What changes under main

A Hugging Face model repository is a git repository, and publishers commit to it after release. On the Phi repositories the observed kinds of change are:

  • Tokenizer and generation config fixes. The most consequential category. A revision that adds a second entry to eos_token_id changes where generation stops — which is precisely the bug described in why a local Phi-3 server misses the stop token. Same weights, different behaviour.
  • Chat template edits. The template lives in tokenizer_config.json. A whitespace change there changes every prompt you render.
  • Modelling code. Phi repositories have shipped custom code loaded under trust_remote_code. Tracking main with that flag set means executing whatever code was pushed most recently.
  • Added or reorganised files. New quantisations, safetensors re-shards, changed filenames — enough to break a loader that expects a particular layout.

Notice what is common to the first three: none of them changes a single weight, and all of them change what the model does. The intuition that “the weights are the model, so a fixed model name is a fixed dependency” is the reason this bites. A checkpoint is a directory of files, several of which are configuration that the runtime obeys, and the configuration is the part that gets patched.

It also matters that a Hugging Face repository is not immutable in the way a package registry is. There is no semantic version to compare, no published changelog, and no notification. The commit history is the changelog, which is exactly why the commit is the right thing to pin.

Find the commit

Every revision has a 40-character SHA. Get the current one from the API rather than copying it out of the web UI:

from huggingface_hub import HfApi

api = HfApi()
info = api.model_info("microsoft/Phi-4-mini-instruct")
print(info.sha)          # the commit main points at right now
print(info.lastModified)

# and the tags and branches available, if the publisher uses them:
refs = api.list_repo_refs("microsoft/Phi-4-mini-instruct")
print([b.name for b in refs.branches], [t.name for t in refs.tags])

Record that SHA somewhere your build reads. A short JSON file next to your dependency manifest is enough, and it makes a model bump a reviewable diff rather than an invisible event:

{
  "models": {
    "generator": {
      "repo": "microsoft/Phi-4-mini-instruct",
      "revision": "0a1b2c3d4e5f60718293a4b5c6d7e8f901234567"
    }
  }
}

Load at that revision

  1. Pass revision to every loader. The model, the tokenizer, and the processor if there is one. Pinning the model and letting the tokenizer track main is the most common version of getting this half-right, and it is the worse half — tokenizer files are what change most often.
  2. State the dtype rather than inheriting it. Not strictly part of pinning, but "auto" resolves differently on different hardware, which undoes some of what you gained.
  3. Treat trust_remote_code as part of the pin. With a revision fixed, the code you execute is fixed too. Review it when you bump.
import json, torch
from transformers import AutoModelForCausalLM, AutoTokenizer

lock = json.load(open("models.lock"))["models"]["generator"]
REPO, REV = lock["repo"], lock["revision"]

tok = AutoTokenizer.from_pretrained(REPO, revision=REV)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    revision=REV,
    torch_dtype=torch.bfloat16,   # explicit, not "auto"
    device_map="auto",
    trust_remote_code=True,       # pinned to this revision's code
)

messages = [{"role": "user", "content": "Name the two end-of-turn tokens Phi-3 uses."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids.to(model.device), max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

Prove the pin took

A pin that silently does nothing is worse than no pin, because you stop looking. Two checks, both cheap:

  1. Download explicitly and compare the resolved commit. snapshot_download returns the local path of the snapshot for that revision, and the directory name under the hub cache is the commit itself.
  2. Run once with the network off. If loading succeeds with HF_HUB_OFFLINE=1, nothing was fetched at import time, so nothing can have resolved to a different revision. If it fails, your pin was being satisfied by a live lookup.
from huggingface_hub import snapshot_download

path = snapshot_download(repo_id=REPO, revision=REV)
print(path)   # .../models--microsoft--Phi-4-mini-instruct/snapshots/<commit>
assert REV in path, "loaded a different revision than the lock file names"
# and, as a separate run:
HF_HUB_OFFLINE=1 python generate.py

Pinning in a serving framework

The same idea, different flag names, and the tokenizer is again a separate argument:

# vLLM
vllm serve microsoft/Phi-4-mini-instruct \
  --revision 0a1b2c3d4e5f60718293a4b5c6d7e8f901234567 \
  --tokenizer-revision 0a1b2c3d4e5f60718293a4b5c6d7e8f901234567 \
  --max-model-len 16384

# text-generation-inference
text-generation-launcher \
  --model-id microsoft/Phi-4-mini-instruct \
  --revision 0a1b2c3d4e5f60718293a4b5c6d7e8f901234567

# Docker: bake the snapshot into the image rather than fetching at boot
RUN python -c "from huggingface_hub import snapshot_download; \
    snapshot_download('microsoft/Phi-4-mini-instruct', revision='0a1b2c3d...')"

The Docker line is the one that turns a pin into a guarantee. A container that downloads at startup can still fail or drift if the repository changes or becomes gated; a container with the weights already inside cannot. It also removes a multi-gigabyte download from your cold-start path, which matters if the thing ever needs to scale out under load.

One caveat specific to serving frameworks: the revision pins the checkpoint, not the framework. A vLLM upgrade can change kernel selection, default sampling behaviour or how the chat template is applied, all with the same pinned weights. Pin both, and bump them in separate changes so that when behaviour moves you know which one moved it.

Mirror it

A pinned revision protects you from the repository changing. It does not protect you from the repository going away — renamed, gated behind an access request, or removed. Phi’s MIT licence explicitly permits redistribution, as the licence page sets out, so mirroring is a decision about storage cost and nothing else.

  1. Download the pinned snapshot with snapshot_download as above.
  2. Copy it to object storage you control, or push it to a private Hugging Face repository under your own namespace, keeping the LICENSE file with it as MIT requires.
  3. Record both the upstream repo-plus-commit and your mirror’s location in the same lock file, so provenance survives the copy.
  4. Point production at the mirror and leave the upstream pin in the lock file as documentation of where it came from.

The one thing a mirror does not preserve on its own is why you chose that commit. Put a line in the lock file, or in the commit message that introduces it, saying what the pin is protecting — “the revision whose generation_config.json lists both end tokens” is a sentence that will save somebody an afternoon in a year’s time. A pin without a reason gets bumped by the first person who wants a newer model and cannot see what would break.

Do this for anything whose exact behaviour you have validated. Between the pin and the mirror, the model becomes the one part of your stack that changes only when you decide it does — which is also what makes the reproducibility work in Phi-4’s determinism at temperature 0 worth attempting at all.

Related