Skip to content
Multigrid

Scaling Laws Interactive Chart

Enter the coefficients of a scaling law, move the compute budget, and read the compute-optimal parameter and token split it implies.

The five coefficients are round placeholders chosen so the curve behaves. They are not any published paper’s fitted values and this page will not pretend otherwise — open the paper you are reading, type its numbers in, and the chart becomes about that fit rather than about nothing.

Predicted loss at the compute-optimal split
1.8720

7.90B parameters trained on 210.9B tokens — 26.7 tokens per parameter. Your split scores 1.8720, which is 0.0000 worse.

Predicted loss against compute, and the penalty for a bad splitloss 1.54 – 4.97 · compute 1e16 – 1e28 FLOPsexcess loss from a bad split · 0.1× to 10× the optimal parameter count0.1× params10× params
Top: predicted loss against compute, both allocated optimally, on a log compute axis. Bottom: how much loss you give up by moving parameters away from the optimum on a fixed budget — note how flat the middle is.
Compute budget
1.0e22 FLOPs
Optimal parameters
7.90B
Optimal training tokens
210.9B
Tokens per parameter
26.7
Predicted loss at the optimum
1.8720
Your parameters
7.90B
Your training tokens
210.9B
Your predicted loss
1.8720
Loss given up
0.0000
Irreducible floor (E)
1.5000
What this assumes: the loss follows L(N, D) = E + A/N^α + B/D^β, and training compute is C ≈ 6ND. Minimising under that constraint has a closed form — N* = (αA·K^β / βB)^(1/(α+β)) with K = C/6 — and that is what the chart plots, exactly, from the five coefficients in the panel. Those coefficients are the whole model: change them and every number moves. The law says nothing about inference cost, which is the reason nobody actually trains at the compute-optimal point any more, and nothing about capability — loss is next-token prediction error, not usefulness.

The lower chart is the one worth the visit. On a fixed budget, the penalty for getting the parameter-to-token split wrong is remarkably flat near the optimum: you can be off by a factor of two or three in either direction and give up very little loss. That is why compute-optimal is a guideline rather than a target, and why the industry moved away from it without much apparent cost — training a smaller model on far more tokens than the optimum is only slightly worse at training time, and much cheaper to serve for the rest of its life.

The upper chart carries the other half. Loss falls with compute, but as a power law with a floor: each equal step along the log compute axis buys a smaller absolute improvement than the last, and the curve is asymptotically pinned at E. There is no compute budget at which the predicted loss reaches zero. Every argument about scaling limits is ultimately an argument about that E term and about how far away it is.

Two cautions before quoting anything from this page. The coefficients are placeholders — the shape is real, the numbers are not, and the tool exists so you can put a paper’s fit in and see what it implies. And a fitted law describes the regime it was fitted on: extrapolating it four orders of magnitude past the largest run in the fit is a hypothesis, not a prediction, however smooth the line looks.

The reasoning behind this

Related tools

Scaling Laws Interactive Chart · Multigrid