Model Editing: Changing One Fact

Model editing changes what a model says about one fact by writing directly to a small number of weights — no retraining, no fine-tuning run, a few seconds of compute. It works well enough to be interesting and fails in ways that are more instructive than the successes.

The problem editing is trying to solve

A model asserts something that has become false, or was never true. A head of state changed. A product was discontinued. A person’s details are wrong. The options are all unattractive: retraining is prohibitively expensive, fine-tuning on the correction risks catastrophic forgetting and needs care to avoid teaching the format rather than the fact, and a system prompt patch is a growing list of exceptions that consumes context on every request.

Editing proposes a fourth: find where the association lives and change it in place. Whether “where it lives” is a coherent notion is the question the rest of this page circles.

The MLP-as-key-value-store idea

The theoretical basis, which predates the editing work: a transformer MLP block can be read as an associative memory. The first (up-projection) matrix computes, for each hidden unit, a score against the incoming residual stream — a set of keys. The non-linearity gates them. The second (down-projection) matrix writes a learned vector for each active unit back into the residual stream — a set of values. So an MLP layer is roughly: match the input against many keys, sum the values of the ones that fire.

If that reading is right, storing an association means having a key that fires on the subject and a value that writes the object. Changing the association means changing the value written for that key, which is a small, targeted, computable modification rather than gradient descent over everything.

Locate then edit, in practice

Meng, Bau and colleagues published this line in 2022 as ROME (rank-one model editing), and extended it in 2023 to MEMIT for editing many facts at once. The recipe:

1. Locate. Run causal tracing over the prompt to find which layers and positions the factual prediction depends on. In the reported results this concentrates at the last subject token in mid-range MLP layers.
2. Compute the key. The activation entering the chosen MLP’s down-projection when the subject is presented. This is the vector the edit will key on, averaged over several prompt phrasings so it is not tied to one wording.
3. Compute the target value. Optimise a vector that, when written at that site, makes the model produce the new object. This is a small optimisation over one vector, not over weights.
4. Solve for the weight update. Find the minimal rank-one change to the down-projection matrix that maps the key to the new value while disturbing its action on other keys as little as possible — a constrained least-squares problem with a closed form.

The alternative family uses a learned editor. MEND (Mitchell and colleagues, 2022) trains a small hypernetwork to transform the gradient of the desired correction into a low-rank weight update, so that editing generalises without a per-edit optimisation. Different machinery, same evaluation problems.

The four axes an edit is scored on

A method reporting only efficacy is reporting the easiest axis. The benchmark shift that made this field honest was the introduction of evaluations built specifically around the last two.

Ripple effects

Cohen and colleagues published an evaluation in 2023 built around the logical consequences of an edit, and the framing is the useful part. A fact in the world is not isolated; it entails others. If you edit X is the capital of Y, then questions about which country contains X, what language is spoken in the capital of Y, and what the capital of Y was before should all move consistently. Existing editing methods handled the edited statement well and its entailments poorly.

Two further failure modes are worth knowing because they bite in any practical use.

• Sequential editing degrades the model. Applying many edits one after another compounds damage; general capability falls off faster than the number of edits would suggest. Batch methods exist partly because of this.
• Edits leak to neighbours. An edit keyed on a subject vector affects nearby vectors, so subjects with similar representations shift too. This is exactly what superposition predicts: you cannot write to one direction without touching everything that shares the subspace.

The result that breaks the story

The locate-then-edit narrative is intuitive: tracing tells you where the fact is, so edit there. Hase and colleagues tested that link directly in 2023 and found it does not hold. Edits succeed at layers that causal tracing does not implicate, and tracing results are a poor guide to where an edit will work best. The localisation step, in other words, is not doing the job the name suggests.

This is the most important finding on the page, and it generalises past editing. It means causal tracing identifies where information flows during a forward pass, which is not the same as where a parameter change has leverage. Both are real, useful facts about the model. They are different facts, and a great deal of writing about interpretability slides between them.

What to do instead, most of the time

For a production system that needs to say something different about a fact, editing is rarely the right tool. The comparison:

• Retrieval. Put the current fact in the context and require the model to ground its answer in it. Auditable, instantly updatable, works across model versions, and testable with the evaluation you already have. The retrieval-versus-tuning comparison covers the trade-offs, and for changing facts retrieval wins by a wide margin.
• Fine-tuning. Appropriate for changing behaviour, format or style across many examples. Expensive and imprecise for one fact, and the failure mode is teaching the model that this kind of question has this answer.
• Editing. Genuinely useful as a research instrument — it tests claims about where and how knowledge is represented — and for offline correction of an open-weights model you control and can re-evaluate. Not for a live system, where the specificity failures would surface as unrelated regressions nobody can trace.