Transformative AI: Economic Definitions of the Threshold

“Transformative AI” is not a fancier word for AGI. It is a deliberate change of subject, from what a system can do to what happens to the world, made by people who had concluded that capability definitions were not doing useful work.

Why capability definitions were abandoned

Anyone trying to make funding or policy decisions under uncertainty needs a threshold they can reason about. Capability definitions turned out to be poor for that purpose, for three reasons that the seven definitions in what would count as AGI make concrete.

They are contested, so a plan keyed to one of them inherits the argument. They are discontinuous with what decisions actually depend on — a system could be superhuman on a battery of tests and change very little, or be unremarkable on tests and change a great deal through sheer deployment breadth. And they resolve on evidence controlled by the people building the systems, which is a poor property for a threshold used in governance.

Defining the threshold by impact avoids all three. It is contested less because impact has established measures. It is directly connected to what decisions depend on, because impact is what decisions are about. And it resolves on public data.

The definition, and its reference points

The formulation that established the term, associated with Open Philanthropy’s research programme, is: AI that precipitates a transition comparable in scale to the agricultural or industrial revolutions. The reference class is chosen deliberately, and it is doing real work.

Both named transitions share a structure. Each changed the growth rate of the economy rather than just its level. Each restructured where people worked and what most work consisted of. Each altered political organisation. And each unfolded over decades — which is the part most often dropped, and the part that makes the definition compatible with gradual change rather than requiring an event.

Attempts to make it quantitative usually go through the growth rate, because that is the one dimension with a long, comparable series. Economic historians estimate that growth per person went from very low rates before industrialisation to substantially higher ones after, an order-of-magnitude change in the rate itself. A threshold defined as a further change of that kind is arbitrary in its exact placement and precise in its type: it names a quantity, a series and a comparison.

PASTA: the mechanism version

Holden Karnofsky introduced a narrower construct: a process for automating scientific and technological advancement. The idea is to identify the specific capability that would produce the transition rather than the transition itself, and the candidate is automation of the research process — because research output is the input to technological progress, and technological progress is what generates growth in the standard models.

The appeal is that it converts a claim about the future economy into a claim about a capability that could be observed and, in principle, measured before its effects arrive. The mechanism it relies on is the one analysed in recursive self-improvement, and the objections there — serial experiment time, evaluation bottlenecks, the un-automatable fraction — apply here unchanged. PASTA is not a weaker version of the intelligence explosion argument; it is the same argument with the growth model made explicit.

The measurement problems it inherits

Trading a capability definition for an economic one buys resolvability and pays for it in lag and in measurement error. Four specific problems, all of which are known limitations of national accounts rather than criticisms invented for this purpose.

• Free goods are undercounted. Output measures count transactions. A service delivered at near-zero price to many users can generate large welfare and little measured product, which is a long-standing critique of how digital goods appear in the accounts.
• The lag is long. General-purpose technologies show measurable productivity effects long after adoption begins, because the complementary investments — reorganised processes, new skills, new capital — take years. Robert Solow’s remark about computers appearing everywhere except in the productivity statistics named the phenomenon; Brynjolfsson and colleagues formalised it as a J-curve, in which measured productivity first falls as intangible investment is made and rises later.
• Attribution is hard. Growth has many causes. Even a large effect arrives entangled with demographics, energy prices, trade and policy, and separating them is contested econometrics rather than observation.
• Growth is not the only transformation. A change that substantially altered political power, or the distribution of income, or the conduct of war, without moving the growth rate much, would be transformative in every ordinary sense and invisible to the threshold. This is the definition’s real weakness and its proponents generally acknowledge it.

How it relates to AGI

The two thresholds are logically independent, and seeing why clarifies both.

Transformative without AGI is entirely possible: narrow systems deployed broadly enough could restructure large sectors with no general intelligence anywhere in the picture. Historical general-purpose technologies did exactly that, and none of them was intelligent.

AGI without transformation is also possible, at least for a period. Capability has to diffuse before it changes anything, and diffusion is bounded by cost, by regulation, by trust, by integration effort and by the availability of the complementary capital that lets an organisation actually use it. A system that could do most cognitive work does not thereby do most cognitive work.

The practical implication for reading forecasts: capability forecasts and impact forecasts are different objects with different lags, and conflating them is the most common error in the genre. What the economic models actually say about the second, and where they disagree, is in AI and economic growth.