Grokking and Delayed Generalisation

Train a small network on modular arithmetic. It reaches perfect training accuracy quickly and stays at chance on held-out data. Keep training long past the point anyone would — orders of magnitude longer — and test accuracy suddenly jumps to perfect. That is grokking, and unusually for a deep learning mystery, the mechanism has been worked out.

The phenomenon

Power and colleagues at OpenAI reported it in 2022. The curve is the whole result: training accuracy hits 100% early, test accuracy stays flat at chance for a very long stretch, and then rises sharply to 100%. Nothing in the training procedure changes at the transition. No schedule fires, no data is added.

It matters because it breaks a reflex. The standard reading of “training accuracy perfect, test accuracy flat” is overfitting, and the standard response is to stop. Here, continuing to train past that point is exactly what produces generalisation. That is a real counterexample to a rule most practitioners apply without thinking, even if the conditions are narrow.

The setting it was found in

Be precise about this, because the conditions do a lot of work.

• Algorithmic tasks. Modular addition, modular division, and similar operations over a small finite group. The training set is a fraction of all possible input pairs and the test set is the rest — so the task has a short exact rule and the data is a complete, noiseless enumeration.
• Small networks. One- or two-layer transformers, or small MLPs. Small enough to analyse exhaustively.
• Small datasets, near a critical fraction. The delay is longest when the training fraction is just above the minimum needed to generalise at all. Give the model most of the table and the delay shrinks.
• Regularisation, usually weight decay. Without it, grokking is much weaker or absent in most reported setups. This is the single biggest clue to the mechanism.

What is actually happening

Nanda and colleagues gave the mechanistic account in 2023 by reverse-engineering a one-layer transformer trained on modular addition. The model does not learn addition in anything like the way a person would write it. It learns a trigonometric algorithm: represent each input as a set of rotations at several frequencies, combine them using angle-addition identities, and read out the answer by finding which output value is consistent across frequencies. The algorithm is exact, and once you know it you can predict the model’s intermediate activations.

The training story that emerges has three phases, and the middle one is the answer to the puzzle:

1. Memorisation. The network fits the training pairs with a solution that stores them. Training loss drops. Nothing generalises.
2. Circuit formation. The generalising algorithm develops alongside the memorising solution, gradually. Training loss is already near zero so it barely moves, and test loss does not move either, because the memorising solution still dominates the output. From the outside, nothing is happening. From the inside, the structure is being built the whole time.
3. Cleanup. Weight decay penalises the memorising solution, which needs large weights spread across many parameters. The generalising circuit is more compact and achieves the same training loss more cheaply. Once it is good enough, weight decay erodes the memorising solution — and the moment that happens, test accuracy jumps.

So the transition is not a sudden discovery. It is the visible moment in a gradual process, and the visibility is an artefact of which solution happens to be dominating the output. That reframing is the most transferable thing about the result.

Progress measures: seeing it before the loss does

The strongest part of the 2023 work is that it produced quantities which move during phase two, when every standard metric is flat. Two of them:

• Restricted loss. Delete everything except the components the generalising algorithm is supposed to use — keep only the relevant frequencies — and measure loss. If the circuit is forming, this improves steadily throughout the plateau.
• Excluded loss. The complement: delete exactly those components and measure what is left. If the memorising solution is being eroded, this degrades steadily.

Both curves move smoothly across a plateau where training and test loss are flat. That is the concrete demonstration that a phase transition in behaviour can correspond to continuous change in mechanism — and it is a template: given a hypothesis about the mechanism, you can build a measure that tracks it and check whether it moves before the capability appears.

The ingredients that make it appear

Does this happen in large models?

The honest answer is that nobody has demonstrated textbook grokking — the flat-then-jump test curve — in a large language model trained on natural text, and there are structural reasons to expect not to. Frontier models are trained for roughly one pass over their data, so the memorise-then-clean-up dynamic has no room to run. Test loss is measured on held-out text from the same distribution rather than on a rule the model must infer. And scaling law curves are famously smooth, which is the opposite shape.

The adjacent claim that does have support is that capabilities can appear sharply as models scale — the emergent abilities literature. That is a different phenomenon, on a different axis (model size rather than training time), and it has its own critique: Schaeffer and colleagues argued in 2023 that many reported emergent jumps are artefacts of discontinuous metrics such as exact-match accuracy, and that smoother metrics on the same tasks show smooth improvement. The emergence debate is worth reading with that critique in hand.

Why it matters even if it does not

Grokking’s importance is not that frontier models do it. It is three things it established that hold generally.

First, the loss curve is a lossy summary of what the network is doing. A flat curve is compatible with substantial internal change. Anyone reasoning about training from loss alone is reasoning from a projection.

Second, memorising and generalising solutions can coexist in the same weights, competing. That is directly relevant to what memorisation means in a large model and to why removing one without the other is hard.

Third, it is one of very few cases where an unexplained empirical phenomenon was explained by opening the model up. That is the existence proof interpretability research needs: a mystery, a mechanism, and a measure derived from the mechanism that predicted something the original observation could not.