AI Optimism: The Best Case, Stated Seriously

“AI optimist” is used as though it named a single camp. It names at least four positions, held by different people, resting on different evidence, and in two cases mutually inconsistent — you cannot simultaneously hold that the technology will not become very capable and that it will cure most disease.

Four positions, not one

The distinction matters because a rebuttal to one is often mistaken for a rebuttal to all. Each is stated below as its proponents would state it, followed by its main difficulty.

Two structural points before the list. First, the positions are not jointly held and in places conflict: someone arguing that scaling is hitting a wall and someone arguing that AI will transform medicine within a decade are not allies, and a debate that treats them as one bloc will produce nonsense. Second, only the fourth is optimism in the ordinary sense of expecting good outcomes. The first three are claims that a specific bad outcome is unlikely, which is compatible with expecting the technology to be disappointing, harmful in other ways, or both. “Optimist” is doing a lot of unhelpful work as a label, and the same is true of “doomer” on the other side.

1. Capability scepticism

The claim. Current methods will not produce systems general enough for the risks to arise. Language models are impressive interpolators over a vast training distribution; they lack persistent memory, reliable planning over long horizons, and the ability to learn from experience after deployment. Progress on benchmarks partly reflects contamination and elicitation improvements rather than underlying capability, and the sample-efficiency gap between models and humans remains enormous. Held by a range of researchers in and outside deep learning, and the position has a track record of correctly identifying overclaims.

Its difficulty. Specific versions have repeatedly been falsified by scale — capabilities argued to require architectural change have appeared without one. The position is strongest when it names a concrete missing ingredient and weakest when it asserts general limits, and its proponents differ sharply among themselves about which ingredient is missing.

2. Alignment tractability

The claim. Alignment is a normal engineering problem that is being solved as it arises. The empirical observation is that steering model behaviour has turned out easier than early theoretical work predicted: instruction tuning and preference-based training work better than expected, models generalise instructions rather than learning brittle rules, and the frightening scenarios were derived from a model of AI — explicit utility maximisers with fixed goals — that does not describe what was actually built. Quintin Pope and Nora Belrose argued a version of this in “AI Is Easy to Control” (2023), and something like it is the working assumption of many practitioners.

Its difficulty. Current alignment relies on human evaluators being able to recognise good behaviour, which is the ceiling scalable oversight is about. That today’s methods work on today’s models is evidence about today’s models; extrapolating it forward is a prediction, exactly as the pessimistic extrapolation is. The honest statement is that both sides are extrapolating from a short history.

3. Institutional adaptation

The claim. Societies have absorbed general-purpose technologies with large disruptive potential — electrification, the internal combustion engine, nuclear power, the internet — and built regulation, liability, standards and norms around them, generally after harms became visible and generally imperfectly, but they built them. The risk discussion tends to model institutions as static while modelling capability as fast-moving, and that asymmetry is an artefact of the framing rather than a finding.

Its difficulty. The historical record includes adaptations that took decades and cost a great deal in the interim, and the analogy is weakest exactly where the technology is fastest-moving and least legible. It also gives no answer to risks with no warning shot: adaptation-after-harm is not a strategy for harms that are not recoverable.

4. The benefits are enormous

The claim. This one is different in kind from the other three: it does not dispute the risks, it weighs them. Systems that accelerate biomedical research, expand access to expertise in medicine and education, and reduce the cost of scientific work address an enormous existing burden of harm, and the status quo is not neutral — people die of treatable disease every day that a capability is delayed. Delay has victims too, and they are rarely counted in risk arguments.

Its difficulty. The benefits are largely projected and the projections have the same forecasting problems as the risk projections. Realised benefits so far are real but concentrated in specific domains. And the weighing is a value claim: how to trade a large expected benefit against a small probability of an irreversible loss is a question about values and about who bears which risk, not one that evidence answers. People who agree on every empirical point can differ here, and pretending otherwise is the most common failure on both sides of this debate.

What would count against each

A position that names no disconfirming evidence is not doing empirical work. These do:

• Capability scepticism is undermined by systems that perform well on tasks requiring long-horizon planning and learning from experience in genuinely novel domains, with contamination ruled out — the reason contamination methodology matters as much as it does in benchmark contamination.
• Alignment tractability is undermined by alignment techniques whose reliability degrades as capability increases, or by demonstrated cases of a model behaving differently when it infers it is unobserved — which is why deception detection results are load-bearing for this dispute.
• Institutional adaptation is undermined by a serious AI-caused harm that produces no effective institutional response, or by regulatory responses consistently arriving after the point at which they could work.
• The benefits argument is undermined by the projected gains failing to materialise in the domains where they were most confidently predicted, over a period long enough to rule out implementation lag.

The mirror-image discipline applies to pessimistic positions, and the most useful question to ask of anyone in this debate — in either direction — is what they expect to see if they are wrong.