Sparse Autoencoders and the Interpretability Papers

Interpretability is the one area of AI research where the papers are genuinely trying to explain rather than to win a benchmark, and it shows in how they are written. It is also an area where a reader can easily come away believing more has been established than has been.

Why you cannot just read the neurons

The obvious approach to understanding what a network has learned is to look at individual units and ask what makes them fire. Do that on a language model and you find that most units are polysemantic: a single neuron responds to an assortment of unrelated things — a piece of legal boilerplate, a chemical name, and a particular verb tense — with no coherent description available.

You could conclude that the network simply has no interpretable structure. The line of work here concludes something more specific and more testable: the structure is there, but it is not aligned to the neuron basis.

The superposition hypothesis

Toy Models of Superposition (2022) makes the case in a setting small enough to be fully understood. If a model needs to represent more features than it has dimensions, and those features are sparse — mostly inactive at any one time — it can store them as overlapping directions in the space rather than assigning one dimension each. Interference between features is real but rare, because they rarely co-occur, and the network tolerates it in exchange for capacity.

The paper demonstrates this in constructed models where the ground truth is known, which is what makes it evidence rather than a story. That design choice is worth admiring on its own terms: when you cannot check your interpretation against reality in the system you care about, build a smaller system where you can.

If superposition is what is happening, then features are directions in activation space, there are more of them than there are neurons, and recovering them requires finding an overcomplete basis in which the activations are sparse. Which is a problem with a known tool.

What the autoencoder does

A sparse autoencoder is trained on a model’s internal activations. It encodes each activation vector into a much wider hidden layer, then decodes it back, with two terms in the loss: reconstruct the original accurately, and keep the hidden layer sparse so that only a few of its many latents are active at once.

activations  ->  encoder  ->  wide sparse latents  ->  decoder  ->  activations'

loss = reconstruction_error(activations, activations')
     + sparsity_penalty(latents)

The hope is that the latents correspond to the features the model is actually using, in a way individual neurons do not. Towards Monosemanticity (2023) applied this to a small transformer and reported latents that were far more consistently interpretable than neurons — each responding to a describable thing — along with evidence from intervening on them.

Two structural notes for a reader. The sparsity penalty and the reconstruction term trade off directly: push sparsity harder and you explain less of the activation, so any reported result sits at a chosen point on that curve and comparisons must hold it fixed. And the method is unsupervised, so “this latent means X” is a hypothesis formed by looking at what activates it, which is exactly the kind of claim that needs an intervention to support it — does suppressing or amplifying the latent change behaviour in the predicted way.

Scaling it up

The obvious question after a demonstration on a small model is whether it survives at frontier scale. Work published from several labs in 2024 addressed this: applying sparse autoencoders to production-scale models and recovering very large numbers of features, including abstract ones, with steering experiments showing that amplifying a feature changes the model’s behaviour in the direction the feature’s description predicts. A widely-shared demonstration of exactly this involved amplifying a feature corresponding to a landmark and watching the model relate everything to it.

Alongside the scaling results came engineering work on the autoencoder itself — architectures that enforce sparsity by construction rather than by penalty, and open releases of trained autoencoders over open models so that researchers without frontier access can work on them. That last development is the one to care about if you want to follow the area, because it is what turns a lab result into something a reader can poke at.

What is not settled

This is the section that a summary of a fast-moving area owes its reader, and the honest version is that several of the central questions are open.

• Whether the latents are the model’s features. A sparse basis that reconstructs activations and looks interpretable is not proof that the model computes in that basis. Distinguishing “a decomposition we find legible” from “the decomposition the network uses” is genuinely hard.
• Feature splitting. Train a wider autoencoder and a single feature can resolve into several finer ones. That is either a hierarchy being revealed or an artefact of the width you chose, and the two are not easy to tell apart.
• Whether it beats simpler baselines downstream. On concrete tasks — detecting a behaviour, steering an output — sparse autoencoders have to be compared against cheaper methods like linear probes, and results reported since 2024 have not been uniformly favourable. This is an active and unresolved comparison, and anyone telling you it is settled in either direction is ahead of the evidence.
• What fraction of the model is explained. Recovering many interpretable features is compatible with leaving most of the computation unaccounted for. Reconstruction error is measurable; coverage of the model’s actual behaviour is not, and papers are generally careful about this distinction while press coverage is not.

Reading advice specific to this area: prefer papers that intervene over papers that only observe. A description of what a latent responds to is a correlation. Turning it up and watching the predicted change is evidence about causation, and it is the difference between an interpretability result and an interpretability illustration.