Superposition: Why One Neuron Means Several Things

Neurons in a language model are usually polysemantic: the same unit fires on academic citations, on Korean text and on nothing you can name. Superposition is the leading explanation — the layer is representing far more features than it has dimensions, by assigning them directions that are not quite orthogonal and living with the interference.

The observation that started it

Pick a neuron in a mid-layer MLP, run a large corpus through the model, and look at the inputs that activate it most. Sometimes you get something clean. Far more often you get a list with no coherent theme, and the honest description is that this neuron participates in several unrelated things.

The lazy conclusion is that networks are simply messy. The superposition hypothesis proposes something more specific and more useful: the network would like a clean one-feature-per-neuron representation, but there are more features worth representing than there are neurons, so it packs them into overlapping directions. The neuron basis is then the wrong basis to look in, and polysemanticity is a consequence of the packing rather than a fact about neurons.

Anthropic set this out in “Toy Models of Superposition” (Elhage and colleagues, 2022). The framing has roots in earlier work on distributed representations and in compressed sensing, but that paper is where the hypothesis got a testable form.

The arithmetic: how many features fit

You can derive the constraint yourself, and it explains why sparsity is the whole story. Assume:

• A layer of dimension d, with m features, each assigned a unit direction.
• Any two feature directions have absolute cosine similarity at most ε. With m > d they cannot all be orthogonal, so ε > 0 is forced.
• On a given input, a fraction s of features are active — the sparsity. A feature is read out by projecting the activation onto its own direction.

Reading feature i gives its own contribution, size roughly 1, plus interference from every other active feature. There are about m·s of those, each contributing up to ε with an effectively random sign, so the interference behaves like a sum of m·s random terms and its typical magnitude is ε·√(m·s). For the readout to be reliable that must stay comfortably below the signal:

ε · sqrt(m · s)  <<  1

so            m  <<  1 / (s · ε²)

worked example
  ε = 0.10        directions 10% off orthogonal
  s = 0.01        1% of features active on any one input

  m  <<  1 / (0.01 × 0.01)  =  10,000 features

  ...in a layer of d = 768 dimensions.

and at s = 0.001 (0.1% active), the same ε gives m << 100,000.

That is the whole idea in one line. Sparsity buys capacity. If every feature were active on every input, s = 1 and the bound collapses to m << 100 — fewer than the dimensions you have, so superposition would be strictly harmful. Language is extremely sparse in this sense: the overwhelming majority of concepts a model can represent are irrelevant to any particular sentence. That is what makes the packing profitable.

The toy model, and what it showed

The experimental setup in the 2022 work is small enough to reproduce in an afternoon. Generate synthetic data with a known number of independent features at a controllable sparsity. Train a tiny autoencoder that must squeeze those features through a bottleneck smaller than the number of features. Then look at what the encoder learned.

The findings that matter:

• Sparsity controls the regime. With dense data the model represents the most important features one-per-dimension and discards the rest. As sparsity increases it starts representing more features than dimensions, accepting interference.
• The transition is not smooth. Features enter and leave superposition in discrete jumps as sparsity varies, rather than fading in.
• The arrangements are structured. Features in superposition organise into recognisable geometric configurations rather than settling into arbitrary positions — pairs pointing in opposite directions, and higher-order regular arrangements as more features share a subspace.

The value of a toy model is that the ground truth is known. You put in a specific number of features and you can check what came out, which is exactly what you cannot do in a real model — and that is why the toy result is evidence about a mechanism rather than proof about GPT-2.

The geometry

The counter-intuitive fact underneath all of this: high-dimensional spaces have far more room than the dimension count suggests, provided you accept “nearly orthogonal” instead of “orthogonal”. At most d vectors can be mutually orthogonal in d dimensions. But the number of vectors you can place with pairwise cosine bounded by a small ε grows exponentially in d — this is the content of the Johnson–Lindenstrauss family of results, and it is why the bound in the section above lands at ten thousand rather than at seven hundred and sixty-eight.

The practical consequence for anyone poking at a model: a feature’s direction is a vector in the layer’s space with no reason to align with any axis. Projecting onto a single neuron measures a smeared combination of everything whose direction has a component there. This is a much better explanation of confusing neuron-activation plots than “networks are complicated”, and it makes a prediction: change basis correctly and the confusion should reduce.

Sparse autoencoders as the proposed way through

If features are sparse and linearly represented but not axis-aligned, then finding them is a dictionary-learning problem: recover an overcomplete set of directions such that each activation is a sparse combination of a few of them. A sparse autoencoder does exactly that. Train a wide layer to reconstruct the model’s activations through a bottleneck with far more units than the activation has dimensions, penalised so that only a handful of units are active per input.

# the objective, in one line
loss = ||x - decode(encode(x))||²  +  λ · ||encode(x)||₁

# x            an activation captured from one site in the model
# encode(x)    a much wider, mostly-zero vector of feature activations
# λ            trades reconstruction fidelity against sparsity

Anthropic published this direction in 2023 and Cunningham and colleagues reported closely related results independently in the same year; the approach has since been applied at production model scale. The reported outcome is that the resulting units are substantially more interpretable than raw neurons — a unit that fires on one recognisable thing rather than five.

The open problem is validation, and it is serious. A sparse autoencoder will always produce a sparse decomposition, because that is what it is optimised to do. Whether the decomposition it finds is the model’s own is a separate question, and the field does not yet have a decisive test. The λ above is a knob that changes how many features you get; there is no principled setting of it that says “this many is correct”. Treat feature counts from this literature as artefacts of a configuration rather than measurements of a model.

What would show this is wrong

Superposition is a hypothesis with content, which is what makes it worth taking seriously. Results that would count against it:

• A model trained with enough width that features could each have a dimension, still showing the same polysemanticity. That would mean packing pressure is not the cause.
• Sparse decompositions that fail to give a causal handle — if intervening on a discovered feature direction does not change behaviour in the way the feature’s description predicts, the decomposition is describing the autoencoder rather than the model.
• Evidence that important features are not linearly represented at all. Superposition assumes a linear representation with sparse activation; a feature encoded non-linearly is outside the frame.

What it means for everything else

If superposition is broadly right, several things follow that shape the rest of this cluster. Neuron-level explanation is attempting to name something that does not have a single name. Ablating a neuron damages every feature that shares its axis, so a clean ablation result is harder to get than it looks. Editing a fact by writing to a direction perturbs everything sharing that subspace, which is one reason edits have ripple effects. And the promising interventions — steering vectors, feature-level ablation — are the ones that operate on directions rather than on units, which is exactly what the hypothesis says you should expect.