Parameter Count as a Marketing Number

For a stretch of years, parameter count was a genuinely good predictor of model quality, and people who used it as a shorthand were right to. Then it stopped working. The interesting part is not that it stopped but which specific changes broke it, because each one is still operating and each one distorts a different comparison.

Why the number used to work

The scaling literature established a real regularity: within a fixed architecture and training recipe, loss falls predictably as parameters, data and compute increase together. That is a strong empirical result and it was not hype. If two models were built the same way on comparable data, the larger one was better, and the size was a reasonable one-number summary.

The conditions in that sentence are the entire story. The regularity holds within a recipe. Comparing sizes across recipes was never what the result licensed, and by the time model sizes became a marketing surface, the recipes had diverged.

It is worth being clear that the shorthand was not naive. For a period when every serious model was a dense transformer trained on comparable web-scale data with broadly similar objectives, the recipe really was close to constant across the field, and parameter count really was the variable that moved. Generations of models were named after their size, leaderboards were sorted by it, and readers who used it as a ranking were reading the situation correctly. The number stopped working because the world changed, not because people were fooled.

Three things that broke it

Sparsity: two numbers where there was one

A mixture-of-experts model activates a fraction of its parameters per token. It therefore has two parameter counts — total and active — and they can differ by an order of magnitude. Compute per token, latency and price follow the active count; memory requirements follow the total. A headline figure that does not say which one it is has told you nothing, and the total is the more flattering number, so it is the one that appears.

Training data quantity and quality

The compute-optimal balance between parameters and training tokens was revised substantially, and the practical effect was models trained on far more data relative to their size. A smaller model trained longer on better-filtered data routinely beats a larger model trained less. Data quality is not visible in any published number, which means the axis that moved most is the axis nobody can compare on.

Post-training and distillation

Preference training, instruction tuning and distillation from a larger teacher all change capability without changing parameter count. A small distilled model can inherit a large fraction of a much larger model’s behaviour on the tasks it was distilled for. Two models of identical size can now differ enormously, which is precisely what a proxy is not allowed to do.

A fourth breaker gets less attention and matters most for anyone comparing hosted models rather than checkpoints: the weights you are served may not be the weights that were released. Numerical precision, quantisation and serving configuration are chosen by the host, and they change output quality without changing any published figure. A parameter count describes a file; what you call is a deployment of that file, and the gap between the two is a real source of quality difference between routes to nominally the same model.

Why the number stopped being published

Most closed frontier models no longer disclose parameter counts at all. The reasons are worth separating, because people tend to assume only the cynical one.

• Competitive. The count leaks information about the recipe and the serving cost.
• Regulatory. Some frameworks key obligations to training compute thresholds, which makes size-adjacent numbers legally relevant rather than merely commercial. See compute thresholds.
• It genuinely stopped being meaningful. Publishing a number that no longer predicts anything invites comparisons that are wrong in both directions.

The consequence is that most confident statements about a closed model’s size are inference from serving latency and price, which are real signals and weak ones. Treat any specific figure quoted for an undisclosed model as an estimate with wide error bars, whoever is quoting it.

What size still tells you

The number is not useless. It has simply moved from being a quality proxy to being an operational one, and for open-weight models that operational meaning is exact.

• Memory. Total parameters times bytes per parameter is the floor for what it takes to hold the weights, which is the first question in whether you can run it at all.
• Speed. Generation is bandwidth-bound, so active parameters set the ceiling on tokens per second for given hardware.
• Within one family, at one moment. The scaling regularity still holds where its conditions hold: the 8B and the 70B of the same release, trained the same way, do rank as expected.

What to compare instead

For a decision about which model to call, the comparable quantities are price per token, latency at your prompt length, and quality on your own examples. All three are observable without anyone disclosing anything, which is the main argument for using them: they do not depend on a vendor choosing to publish a number, and they cannot be quietly redefined between releases.

There is one context where the size question is still the right question, and it is worth naming so the page does not overcorrect: if you are running the model yourself, size is the constraint, because it determines the hardware. There the number is not marketing at all — it is a specification, and what a small model can actually do is a live and useful question precisely because the answer has turned out to be more than the scaling shorthand would have predicted. The claim of this page is narrow: size stopped being a quality ranking across models built differently. It never stopped being a fact about what it takes to run one.