Why Models Have a Knowledge Cutoff (And What Leaks Past It)

The mental model most people carry is a wall: everything before the date is known, everything after is not. The truth is a gradient in one direction and a leak in the other, and both have specific causes worth knowing before you rely on a model for anything time-sensitive.

What a cutoff actually is

A model’s weights are frozen when training ends. Nothing is added afterwards — a conversation does not update anything, and the model has no state between calls. So the knowledge baked into the parameters is bounded by the last document in the training corpus.

But “the last document” is not one date. A corpus is assembled from web crawls, licensed collections, code repositories, books and curated sets, each with its own collection date, and then filtered and deduplicated. The published cutoff is a summary of a pipeline, and summaries lose detail.

Four reasons it is not a wall

• Coverage thins out before it stops. The internet discusses an event for years after it happens: the analysis, the retrospectives, the corrections, the encyclopedia edits. A crawl taken in month X contains the full accumulated discussion of events from three years ago and only the first flurry about last month. So recall degrades gradually as you approach the cutoff rather than ending at it. Recent-but-in-range facts are the ones a model gets thinly and confidently wrong.
• Training is multi-stage and the stages have different dates. Pretraining, continued pretraining, supervised fine-tuning, preference data and safety data are collected at different times, sometimes months apart. Facts can enter through a late stage — often exactly the facts someone thought were worth correcting.
• Old documents discuss scheduled future events. A 2022 article can name the host city of a 2028 tournament, the date of an election, the sunset date of a standard, a published product roadmap. Post-cutoff facts of this kind are genuinely in the training data, which is why a model can be right about a future date and wrong about last week.
• Deduplication and filtering distort the boundary. Near-duplicate removal preferentially thins heavily-syndicated recent news; quality filters treat freshly-published pages, which have fewer inbound links and less accumulated signal, differently from established ones. Both bias the corpus against the newest material over and above the crawl date.

Why the model cannot tell you

Asking a model for its own cutoff produces an answer, and the answer is not a measurement. The model cannot inspect its corpus; it has no access to anything but the forward pass it is running. The response comes from one of three places: a date written into the system prompt by the provider, a statement included in the fine-tuning data, or a guess shaped by the distribution of dates the model saw.

That third route produces a specific, well-known error: models often understate their cutoff, because material about a period keeps accumulating after it, so the most recent months are underrepresented and “feel” less familiar than earlier ones. Treat the self-reported date as a weak signal, and the provider’s documentation as the real one.

What legitimately gets past it

Everything that arrives in the context window, because the context is not memory — it is input. Grounding a model in retrieved documents, giving it a search tool, or simply pasting today’s data into the prompt all bypass the cutoff entirely for the facts you supply. This is the actual fix, and it is worth stating plainly because the alternative — waiting for a model that knows — never arrives for anything that changes weekly.

What does not get past it: anything you did not supply. The failure that costs the most time in practice is library and API versions. A model will confidently write code against the interface that dominated its corpus, and confidently explain a parameter that has since been renamed, because a plausible-looking continuation is exactly what it is built to produce — the mechanism behind hallucination generally.

Probing it yourself

No numbers are reported here, and that is the point: a table of what some model knew about some month is about one model on one day, and it is worthless for the model you are about to deploy. The probe below is the method, and you supply the facts, because the facts have to be ones whose first-publication date you can verify.

# Build 8-12 facts per month across a band around the claimed cutoff.
# Each needs: a question, a verifiable answer, and the date the answer
# FIRST became publicly available. Events, releases, version numbers
# and appointments work well; anything discussed in advance does not.

PROBES = [
  # (month, question, accepted answers)
  ("2024-09", "Which version of <library> introduced <feature>?", ["4.2"]),
  # ... your own, with sources you checked
]

# Ask each with no retrieval, no tools, temperature 0, and a strict
# instruction to answer "unknown" rather than guess. Score three buckets:
#   correct / wrong / abstained
# Plot the three rates by month. The shape you are looking for is the
# gradient described above: correctness falling and abstention rising
# over several months, not a cliff on one date.

Two design rules make the result mean something. Exclude anything that was scheduled or discussed in advance, or you will measure the third mechanism above rather than the cutoff. And score abstention separately from error: a model that says “I do not know” past its cutoff is behaving well, and collapsing that into “wrong” hides the single most useful property you are testing for.