Does an LLM “Understand” Anything?

The argument about whether language models understand anything is unusually stable: the same two positions, restated for years, with neither side moving. That is a symptom. When an argument does not converge, the usual cause is that the word in the middle of it is doing several jobs at once.

Three questions wearing one word

“Understand” covers at least three claims that come apart cleanly, and a great deal of the disagreement is one person affirming the first while another denies the third.

Notice that these can take different values. A pocket calculator scores well on the first for arithmetic and nobody thinks it has the third. A person with a head injury may have the third intact and the first badly impaired. Bundling them into one word guarantees that any answer is wrong in one of the senses.

Behavioural: does it generalise?

This is the sense that matters for anyone deciding whether to put a model in front of a customer, and it is settled the same way it would be for a person: give it cases it cannot have seen and check whether the competence holds.

The measurement is harder than it sounds for one specific reason. Training corpora are enormous and largely undisclosed, so a test the model does well on may simply be a test it has already read. Benchmark contamination is not a hypothetical risk here; it is the default assumption for any public test set old enough to have been crawled. Evidence for behavioural understanding therefore has to come from held-out cases constructed after the training cut-off, or from tasks whose surface form is varied in ways that leave the underlying problem identical.

Both of those have been done, and the result is genuinely mixed rather than a win for either side. Models solve novel problems that were not in any corpus, including problems in formats invented specifically to be new. They also show a pattern that is hard to reconcile with full understanding: performance that moves substantially when a problem is restated in different words, different variable names or a different order, without any change to what is being asked. That is prompt sensitivity, and a person who understood the problem would not exhibit it to anything like the same degree.

Representational: is there a model inside?

The strongest evidence in this argument comes from probing work, and it is worth stating precisely because it is often overstated in both directions.

The method is: train a small classifier to read some property of the world off the model’s internal activations. If a model trained only on sequences of board-game moves carries activations from which the current board position can be decoded — and, more tellingly, if editing those activations changes the model’s subsequent moves in the way the edited board would predict — then something inside it is tracking board state rather than move-string statistics. Results of that shape exist and are reproducible.

What they license is narrower than the headline. A decodable property may be a correlate rather than the thing itself, and a probe with enough capacity can find structure that the model never uses. Causal interventions of the kind above address that objection directly, which is why they carry more weight than decoding alone. The honest summary is that internal world-tracking has been demonstrated for constrained domains, that it is patchy and domain-specific rather than general, and that nobody has shown the same for open-ended natural language, where there is no clean ground truth to probe for.

The strongest objection

The best version of the case against is not “it is just statistics”. It is an argument about what the training signal could possibly contain, and it goes like this.

Meaning is a relation between expressions and things outside language. A system trained purely on the co-occurrence of forms has access only to the left-hand side of that relation. It can learn everything about how words distribute with respect to each other and still have no access to what they refer to, because the referents were never in the data. On this account, a model can be arbitrarily good at predicting form and still not be doing semantics at all — and its successes are explained by the fact that form carries an enormous amount of information about meaning, not by the model having acquired meaning.

This is a serious argument made carefully by people who work on language for a living, and the cheap replies to it do not work. The replies that do engage are: that multimodal training puts referents in the data, which weakens the premise without eliminating it; that human language learning is also heavily distributional; and that “grounding” may admit of degrees rather than being present or absent. None of those is a refutation. They relocate the disagreement to how much grounding is enough, which is a better place for it to be.

Where that leaves the answer

• Behavioural: partly, and unevenly. Real generalisation to genuinely novel problems, alongside failures that no understanding system should have. Both are true at once and any summary that reports only one is selecting.
• Representational: some, in narrow domains, with real evidence. Stronger than the dismissive view allows, much weaker than “it has a world model” implies.
• Phenomenal: no evidence either way, and no method. Anyone confident here is confident about something they cannot check. See why sentience claims keep recurring.

The practical consequence: for any decision you are actually making, replace “does it understand” with “does the competence survive the variation my users will produce”. That question has a method, and the method is a held-out set of your own rather than a position in a debate.