Alignment Research: A Map of the Agendas

Alignment research is hard to follow partly because the work is technical and mostly because the participants disagree about what the problem is. A map organised by institution tells you nothing. A map organised by what each approach assumes tells you why the same evidence moves different people in different directions.

Two axes that organise the field

Nearly every agenda can be located on two questions, and locating it explains most of what it does.

First: is the difficulty specifying the objective, or conveying it? One view holds that we cannot write down what we want with enough precision, so any objective we optimise will be satisfied in ways we did not intend. Another holds that the trained system may pursue something other than the objective it was trained on, because training selects for behaviour on the training distribution and not for a particular internal goal. These are usually distinguished as outer and inner alignment, and an approach that assumes the first is the real problem will look wasteful to someone who assumes the second.

Second: do current systems tell us about future ones? If yes, the productive move is empirical work on models you can actually run. If no — if the concerning behaviours only appear at capability levels not yet reached — then experiments on today’s systems risk being studies of a different phenomenon, and theoretical or conceptual work is the better investment. This axis, more than any technical dispute, predicts what a given researcher spends their time on.

The agendas

Learning from human feedback, and scaling it

The mainstream production approach: collect human preferences, fit a model of them, optimise against it — the pipeline described in the InstructGPT paper. The known problem is that it is bounded by what human raters can evaluate, and the obvious extension is to have models assist with the evaluation, as in the Constitutional AI method. Scalable oversight is the umbrella term for approaches in this direction: debate between models, recursive decomposition of a hard judgement into easier ones, and model-assisted critique.

Interpretability

Understand the computation rather than only the behaviour, on the grounds that a system you cannot inspect cannot be trusted for reasons beyond “it behaved well when we watched”. The current technical frontier is feature decomposition, covered in the sparse autoencoder papers, alongside circuit-level analysis of how specific behaviours are implemented. The agenda’s appeal is that it aims at evidence rather than at reassurance; its difficulty is the gap between recovering interpretable structure and explaining the model’s actual decisions.

Evaluations for dangerous capabilities

Rather than making systems safe, measure whether a specific system has capabilities that would be dangerous, and gate deployment on the result. This is the agenda with the clearest connection to policy, since a threshold you can test is a threshold you can regulate against. Its central technical problem is that an evaluation establishes a lower bound: failing to elicit a capability is weak evidence it is absent, because elicitation is itself a skill and improves after release.

Control rather than alignment

A deliberately pessimistic framing: assume the system may be pursuing something you did not intend, and design the deployment so that it cannot cause serious harm anyway — restricted permissions, monitoring, untrusted-model protocols, systems that check each other. The attraction is that it does not require solving alignment; the cost is that it requires the containment to hold, which is an engineering claim about systems under adversarial pressure.

Theoretical and conceptual work

Work on what the problem even is: formal treatments of agency and optimisation, why a system might resist modification, what it would mean for a goal to generalise correctly out of distribution. Frequently criticised for not producing testable predictions, and frequently vindicated in the sense that its vocabulary — specification gaming, instrumental goals, deceptive alignment — is what everyone else uses to describe empirical findings.

What each one is betting on

Read across that column and the structure of the field becomes clearer: these are not competing solutions to one problem. They are different bets about which part of the problem is binding, and most of them are compatible with each other.

Where the real disagreements are

• Timelines. Not a technical disagreement about alignment at all, but it drives everything downstream — whether to invest in work that pays off in ten years, and how much risk to accept from deploying now.
• Whether current behaviour is evidence. When a model exploits a flaw in its reward, one camp reads a preview of a serious future failure mode and another reads an ordinary bug. Same observation, opposite updates.
• Whether capability and alignment come together. If more capable systems understand human intent better, some of the problem dissolves with scale. If capability and the disposition to pursue what we meant are independent, none of it does.
• Whether the risk is misuse or misalignment. Capable systems used deliberately for harm and systems pursuing unintended goals require almost entirely different mitigations, and arguments often stall because the two are not separated.

Reading this literature

Two practices make it navigable, and both are the general reading habits of this cluster applied to a field where the stakes make them harder to maintain.

First, separate the demonstration from the framing. Many alignment papers contain a concrete, checkable experimental result wrapped in a motivation about long-term risk. The experiment is evidence and can be evaluated on the terms of any other empirical paper — baselines, ablations, how many runs. The motivation is an argument, and you can accept the result while disagreeing with what it portends. Conflating the two is why discussion of this work is unusually bad.

Second, notice when a demonstration was constructed. A paper showing that a model behaves deceptively under conditions the authors deliberately engineered has shown that the behaviour is possible to produce, which is genuinely informative and is not the same as showing it arises naturally. Good papers in this area are explicit about the distinction. Coverage of them frequently is not, and the gap between the paper and the headline is wider here than almost anywhere else in machine learning.