Emergent Abilities: The Mirage Debate

This is one of the few disputes in the field where a sceptical argument was made precisely enough to be checked, and where checking it changed what people meant by the original claim without simply refuting it. It is worth understanding as a model of how these arguments should go.

The original claim

Researchers observed that on certain tasks, model performance stays at chance across a wide range of model scales and then rises sharply past some threshold. The term adopted for this was emergent abilities: capabilities absent in smaller models and present in larger ones, appearing discontinuously rather than improving gradually.

The claim was consequential rather than decorative. If capabilities appear without warning at scale, then evaluating a model at one size tells you little about what the next size will do, and the safety argument for pre-deployment testing gets much stronger. It also suggested something qualitative rather than merely quantitative was happening as models grew.

It is worth noting that the original observation was not a marketing claim. It came out of systematic evaluation across many model sizes, the curves were published, and the researchers were explicit that they did not have an explanation for the shape. That is the normal way an anomaly enters a literature, and the phrase “emergent” was descriptive of the curve rather than an assertion about what caused it. Much of the subsequent heat came from the word being read as an explanation when it was a label for something unexplained.

The counter-argument

A subsequent line of work — most prominently a paper asking whether emergent abilities are a mirage — proposed a deflationary explanation: that the discontinuity lives in the measurement rather than in the model. Specifically, that emergence appears when the chosen metric is discontinuous or nonlinear, and largely disappears when a continuous metric is used on the same task and the same models.

What makes this a good argument rather than a rhetorical one is that it is a prediction. It says: take the tasks where emergence was reported, rescore the same outputs with a smoother metric, and the sharp jump should flatten into a gradual curve. That is checkable, and checking it is what the debate then consisted of.

The mechanism, worked through

The clearest case is a task scored by exact match on a multi-step answer — multi-digit arithmetic, for instance. Suppose a model must produce five tokens correctly and gets the whole item wrong if any one is wrong.

Illustrative. Per-token accuracy improves smoothly with scale;
exact-match accuracy is that value raised to the fifth power.

  per-token   0.50   0.70   0.80   0.90   0.95   0.98
  exact-match 0.03   0.17   0.33   0.59   0.77   0.90

The underlying skill improves in even steps.
The reported score is flat, then leaps.

Nothing discontinuous happened to the model. A smooth improvement in the underlying quantity was passed through a function that compresses low values toward zero and then rises steeply, and the shape of the reported curve is the shape of that function. The same effect arises from any all-or-nothing scoring rule, and from multiple-choice scoring where a model must exceed a threshold before it beats chance at all.

Continuous alternatives — token-level edit distance, per-step accuracy, log-probability assigned to the correct answer — show the improvement the whole way up. This is the same family of concern as choosing the metric before the result, and it is a general lesson rather than a fact about emergence.

What survives the counter-argument

The deflationary account is strong and it is also narrower than the headline “emergence was a mirage” suggests. Three things it does not establish:

• Not all reported cases are metric artefacts. The argument explains a large share of them cleanly. Whether it explains every case, including ones on tasks with naturally continuous scoring, remains contested in the literature.
• Smooth underneath does not mean predictable. If the usable capability is the exact-match one — and for most deployed applications it is, because a partially correct answer is a wrong answer — then the thing you care about really does appear suddenly, even though the underlying quantity moved smoothly. The unpredictability the safety argument rested on is reduced, not eliminated.
• It says nothing about what capabilities exist. The argument concerns the shape of the curve, not its endpoint. A model that can do something it could not do at smaller scale can still do it.

Why this matters outside research

Two practical consequences, both of which apply whether or not you care who won.

First, when a launch post shows a capability appearing suddenly, the metric is part of the claim. A jump on an exact-match score is consistent with a modest underlying improvement, and a jump reported on a continuous metric is a stronger result. This is one of the more useful things to know when reading a launch chart.

Second, for your own evaluations: if you score pass or fail on multi-step tasks, expect flat-then-jump curves and do not read them as the model suddenly acquiring a skill. Track a continuous quantity alongside the binary one and you will see improvement coming rather than being surprised by it. That is a cheap change and it makes model comparisons far more informative.

The wider lesson is the one worth carrying out of this dispute. A reported curve is a composition of two things: how the system changed, and how the measurement transforms what it sees. Those are separable, and separating them is often more informative than either the original finding or its rebuttal. The reason this particular exchange counts as a good one is that both sides made claims specific enough to be checked against the same outputs — which is rarer in this field than it should be, and is the standard the rest of evaluating an AI claim is trying to reach.