Algorithmic Efficiency: Progress Without More Compute

Hardware is not the only thing getting cheaper. The compute required to reach a fixed level of capability falls over time as methods improve, and that second curve is measurable — with more caveats than the headline versions carry.

The question the measurement answers

Fix a capability: some stated performance level on some stated task. Ask how much training compute was needed to reach it in each year since it first became reachable. If that quantity falls, the fall is algorithmic progress, because the hardware cost of a FLOP is held out of the comparison entirely.

The framing matters. This is not a measure of how good the best system is, and it is not the hardware price-performance curve in compute trends. It is the efficiency with which the field converts compute into a fixed capability, and it is the term that makes a compute forecast insufficient on its own: if efficiency improves at a meaningful rate, effective compute grows faster than physical compute.

How it is measured

1  pick a task and a performance threshold on it
2  collect systems that reached the threshold, with the
   training compute each used, and the date
3  take the frontier: the least compute achieving the
   threshold at each point in time
4  fit a curve to that frontier over time
5  read off the doubling time of efficiency, usually
   reported as a halving time for required compute

Step 3 is what makes it a measure of the field’s best practice rather than of the average paper. Step 1 is where nearly all the sensitivity lives, and step 2 is limited by the fact that compute figures for many systems are themselves reconstructions with error bars.

The named work

Three lines of research are worth reading in the original if this quantity matters to your reasoning.

• Hernandez and Brown, 2020. The paper that established the format, measuring the compute needed to reach a fixed image-classification accuracy over several years of published models. It is also the clearest statement of the method’s limits, by the authors themselves.
• Erdil and Besiroglu, 2022. Algorithmic progress in computer vision, with a formal decomposition separating gains attributable to better methods from gains attributable to more compute and more data — which is the hard part, since real systems change all three at once.
• Language-model efficiency work, from 2024 onward. Applying the same approach to language models, where the target is a fixed loss or a fixed benchmark level rather than a classification accuracy. Harder, because the compute figures are less well documented and the capability measures are less stable.

Their headline halving times are deliberately not quoted here. Each is specific to a domain, a threshold and a period, the papers report confidence intervals that are wide, and the number quoted without those three qualifiers is the form in which it is almost always repeated and almost always misused. Go to the paper; the interval is on the page.

What the number is sensitive to

• The threshold. Efficiency gains are usually larger for capabilities that were once expensive and are now routine than for capabilities near the current frontier. Measuring at a low threshold flatters the rate; measuring at a high one has too few data points.
• The task. Rates differ across domains, and the domains with the cleanest measurements are the ones with long-lived standard benchmarks — which is a selection effect, not a representative sample.
• Hindsight. Reaching an old threshold today benefits from knowing it is reachable and from architectures designed after the fact. Some measured efficiency is knowledge of the answer, and separating that from method improvement is not straightforward.
• Hardware-software entanglement. Some gains come from techniques that exist because of hardware features — attention implementations designed around memory hierarchy, low-precision training supported in silicon. Whether those count as algorithmic or hardware progress is a classification decision that moves the number.
• Data quality. Better curation reduces the compute needed for the same capability. Counting that as algorithmic progress is defensible and is a choice, and it interacts directly with the supply question in the data wall.

Where the gains come from

Decomposing the improvement is more informative than the aggregate rate and is much less often attempted. The identifiable sources fall into four groups.

Architecture — attention mechanisms, normalisation placement, activation functions, and sparsity, where mixture-of-experts routing reduces the compute per token for a given parameter count. Training procedure — optimisers, learning-rate schedules, batch-size scaling, and the compute-optimal sizing results themselves, which were pure efficiency gains obtained by allocating a fixed budget differently. Data — deduplication, filtering, curriculum ordering and mixture weights. Post-training — instruction tuning, preference optimisation and distillation, which can bring a small model to a capability that previously required a large one, and which the standard measurement does not always separate cleanly from pretraining efficiency.

Why it changes forecasts

The practical consequence is that effective compute is the product of two growth rates, not one. A forecast built on hardware and capital alone understates the trajectory if algorithmic progress is substantial, and any capability-threshold argument is therefore sensitive to a parameter that is measured much less precisely than the compute series is.

It also complicates governance instruments keyed to training compute. If the compute needed for a given capability falls steadily, a fixed FLOP threshold catches fewer systems each year without anyone changing it — a drift that is structural rather than accidental, and one of the design problems described in compute thresholds.

And it is a live consideration in the argument about whether progress is bottlenecked. Efficiency gains are a partial substitute for the constraints in compute trends and the data wall: a field approaching a physical limit on one input can keep moving by needing less of it. How far that goes is unknown, and it is one of the places where an honest forecast has to widen its interval rather than pick a side.