World Models and Physical Understanding

“It does not really understand, it is just predicting tokens” is a claim about internal structure, and internal structure is something you can go and look at. The results are more interesting than either side of the slogan.

Three senses of the term

• The reinforcement-learning sense. An explicit learned component that predicts the next state of an environment given an action, used for planning. Ha and Schmidhuber’s 2018 paper made the term current in this sense: a model trained to simulate an environment so an agent can be trained inside its own simulation. Precise, and it names a specific module.
• The representational sense. Any internal structure that mirrors the state of the domain being described, whether or not it was trained for. This is the sense in which people ask whether a language model has one, and it is a claim about what the activations encode.
• The philosophical sense. Understanding, grounded reference, knowing what a word is about. The oldest version of the question and the one with no measurement procedure attached, which is why the rest of this page uses the second sense.

Arguments about whether language models have world models are frequently arguments in which one party means the first sense — a module that plainly is not there — and the other means the second.

An operational definition

To make the representational claim checkable, state what would have to be true. A system has a world model of some domain if there is an internal state that satisfies three conditions.

1  DECODABLE   the domain state can be read out of the activations
               by a simple decoder trained on them
2  CAUSAL      editing that internal state changes the model's
               behaviour in the way the domain implies
3  GENERAL     it holds on states never seen in training, not only
               on memorised configurations

Condition 1 alone is weak: a powerful probe can find structure that the model does not use, so decodability on its own shows only that the information is present somewhere. Condition 2 is what upgrades a correlation into a claim about mechanism, and it is testable by intervention rather than observation. Condition 3 is what separates a model of the domain from a lookup table.

What probing has shown

The clearest result of this shape is the Othello work. A transformer trained only on sequences of legal moves in Othello — never shown a board, never told the rules — was probed for the board state, and the state was decodable from its activations. Li and colleagues published the original result in 2023; Neel Nanda subsequently showed the representation is recoverable with a linear probe, which matters because a linear probe is much less capable of manufacturing the structure it finds.

The important part is the intervention. Editing the internal board representation changed the model’s subsequent move predictions in the way the edited board implies. That satisfies condition 2 in a specific, narrow domain: the representation is used, not merely present.

Related findings have accumulated. Linear representations of properties such as spatial and temporal features have been recovered from language model activations. Chess-trained transformers show board-state structure. The broader mechanistic interpretability programme — features, circuits, sparse autoencoders — supplies the methods, and without them none of this would be checkable at all.

What probing cannot show

Three limits, all of which are stated by the researchers doing the work and are frequently dropped when the results are summarised.

Domain scope does not generalise. Othello is a small, fully observable, deterministic domain with a compact state. Evidence that a model builds a representation of it is evidence about that kind of domain. It is not evidence that a frontier model has a representation of physics, or of a person’s beliefs, or of a codebase — those may hold, and would need their own demonstrations.

Probes can find structure that is not used. The stronger the probe, the more it can extract from activations that the model itself does not read out. This is why the intervention step is not optional, and why linear probes are more informative than powerful ones despite finding less.

Partial models are the normal case. The interesting finding in this area is not a binary. Systems appear to have representations that are locally coherent and globally inconsistent — right about a fact in one framing, contradicting it in another. That is a different situation from both “has a world model” and “just does surface statistics”, and it fits the observed behaviour better than either.

Physical understanding specifically

Physical prediction is where the gap is most visible, and it is measured directly. Benchmarks in this area — IntPhys, PHYRE, Physion and their successors — test whether a system predicts the outcome of a physical scene: whether a stack falls, where an object goes when it leaves view, whether a scenario violates object permanence. They are designed so that surface statistics do not suffice, typically by pairing physically possible and impossible versions of visually similar scenes.

Video generation models complicate the picture rather than settling it. Systems that produce long, coherent video have been described by their developers as world simulators, and the visual coherence is real. The counter-observation is also real: generated video exhibits violations of object permanence and material behaviour that a system with a physical model would not make. What follows is disputed — whether these are artefacts of the generation process that will disappear with scale, or evidence that appearance is being modelled without physics. This is an open empirical question and worth flagging as one rather than resolving in either direction.

Where the disagreement stands

Yann LeCun’s position is that predicting observations directly is the wrong objective, because most of the observation is unpredictable detail that consumes capacity; his proposed alternative, developed in the 2022 autonomous machine intelligence position paper, is to predict in a learned abstract representation instead, which is what the joint embedding predictive architecture line of work implements. The scaling position is that adequate world models arise from prediction at sufficient scale and diversity, and points at the probing results as early evidence for exactly that.

Both are consistent with what has been demonstrated so far, which is the honest state of it. The evidence establishes that internal representations of domain state can and do form without being designed in, in domains simple enough to check. It does not establish that this extends to the physical world, and it does not establish that it cannot.