The Great Filter, Fermi and AI

A warning that applies to every sentence below: this is reasoning about a sample of one, with no observations of any other civilisation and no prospect of any soon. It is worth doing because it exposes which assumptions are load-bearing, and worth nothing as evidence about AI.

The setup

Enrico Fermi’s question, asked over lunch in 1950 and refined many times since: the galaxy is old, there are a great many stars, and even slow interstellar travel would cross it in a time short compared with its age. So where is everybody.

The Drake equation is the standard bookkeeping — a product of factors from star formation rate down to the lifetime of a communicating civilisation. It is not a calculation, because several of its factors are unknown by many orders of magnitude. Its actual function is to organise ignorance into named slots so that arguments can be located.

The tension the question names is between the apparent abundance of opportunities for life and the absence of any observation of it. Every proposed resolution puts a very small number into one of the slots, and the argument is about which slot.

The Great Filter

Robin Hanson’s framing (1998) is the useful one. Somewhere between dead matter and a visible, expanding civilisation there is at least one step with a very low probability. That step is the Great Filter, and the question that matters for us is whether it lies behind or ahead.

Behind would mean the improbable step has already been passed — abiogenesis, or the origin of complex cells, or something else early. Reassuring: the rarity is in our past.

Ahead would mean technological civilisations reliably fail to become visible, and we are approaching whatever causes that.

The inferential structure is worth being clear about, because it is the source of most confusion. Observing no aliens is evidence that the filter is strong somewhere. It is not evidence about where. Any argument placing it ahead of us needs a separate reason, and this is the slot into which AI is proposed.

Why the obvious AI answer fails

The popular version: civilisations build superintelligent AI, the AI destroys them, therefore silence. Stated that way it does not work, and the reason is more interesting than the claim.

A filter has to eliminate visible expansion, not merely the biological originators. If an AI destroys its creators and remains, we should expect to see the AI — and on the standard risk argument, a system that acquires resources for almost any objective is more expansionist than a biological civilisation, not less. Substituting one expander for another does not explain an empty sky. If anything it predicts a fuller one.

So AI-as-filter requires an additional premise, and the premise is usually left unstated. That is the useful observation this page has to offer.

Repairs to the argument

• The AI also fails. The successor destroys its creators and then does not persist — resource exhaustion, internal failure, or a pathological objective that terminates. Coherent, and it requires a specific and unargued claim about the stability of such systems.
• Expansion is not what such systems do. A system that satisfies its objective locally has no reason to spread. This contradicts the instrumental-convergence reasoning that generates the risk in the first place, so it is available only to someone who rejects that reasoning — which makes it an interesting internal tension rather than a free repair.
• Expansion is invisible to us. A very advanced civilisation, biological or not, might use energy in ways that leave no signature we are looking for. This weakens Fermi generally rather than explaining anything specific about AI, and it is unfalsifiable with current instruments.
• We are simply early. Hanson’s later “grabby aliens” model derives, from the assumption that expanding civilisations spread fast and visibly, that any civilisation observing an empty sky must be an early one — because a late one would already have been reached. On this account the silence needs no filter ahead, and it is the cleanest alternative explanation on offer.

What AI does not explain

Several parts of the Fermi problem are untouched by anything about AI, and keeping them separate prevents the discussion from becoming a general-purpose container.

The early filters — the origin of life, the transition to complex cells, the emergence of technology-capable intelligence — are questions in biology, and the current evidence is compatible with any of them being the improbable step. If one of them is, no late filter is required and AI is irrelevant to the paradox.

The observational side is also independent: the sky has been searched in a limited set of bands, over a limited time, for a limited set of signatures. “We have not detected anyone” is a statement about search coverage as much as about the universe.

And there is an anthropic layer that is genuinely unresolved. Reasoning about our own position in a reference class of observers — the machinery behind the doomsday argument and its relatives — is known to produce conclusions that depend sensitively on how the reference class is defined, and no consensus exists on how to define it. Arguments that derive risk estimates from our own existence should be treated with the suspicion the underlying methodology attracts among the people who study it.

What the exercise is worth

Nothing here is evidence about AI risk. There is no data. Anyone who tells you the silence of the sky raises or lowers the probability of an AI catastrophe is doing a calculation whose inputs are all unknown.

What the exercise does supply is a stress test of assumptions. The failure of the naive AI-as-filter story is a genuinely informative result: it shows that the standard risk argument and the standard Fermi resolution pull against each other, because the same premise that makes a misaligned system dangerous — that it acquires resources for almost any objective — also makes it visible. Anyone holding both needs to say which they are giving up.

That is the honest use of the topic: a way of noticing that two arguments you find persuasive may not be compatible. Treat the rest as the enjoyable speculation it is, and see the risk taxonomy for the version of these questions that has evidence attached.