What Forecasting 10,000 SKUs a Night Actually Costs

There is no public price list for “forecast 10,000 SKUs”, and any figure presented as one is guesswork wearing a decimal point. What can be done honestly is to state the inputs, do the arithmetic in the open, and let you substitute your own numbers. The arithmetic is the part worth keeping.

Four assumptions, stated up front

Everything below is derived from these four values. Each is an assumption, not a measurement, and each is the one you should replace first with a figure from your own environment:

• 10,000 series, forecast once per night at a 28-day horizon.
• 1.2 seconds of one CPU core per series to fit and predict an automatic ARIMA on three years of daily history. This is the assumption most worth measuring yourself: it varies by an order of magnitude with series length, the search space the implementation explores, and whether seasonal orders are enabled.
• A 32-core instance doing the work, with the fits distributed across cores.
• $1.50 per hour for that instance, and $2.00 per GPU-hour for the neural comparison. Assumed rates, quoted here only so the multiplication has something to multiply.

The classical run, costed

total core-time   = 10,000 series × 1.2 s      = 12,000 core-seconds
                                                = 3.33 core-hours

wall clock        = 12,000 s / 32 cores        = 375 s  ≈ 6.3 minutes
                    (at perfect parallel efficiency)

instance cost     = (375 / 3600) h × $1.50/h   = $0.156

annual, nightly   = 365 × $0.156               ≈ $57

Sixteen cents a night. Fifty-seven dollars a year. That is the whole compute bill for automatically fitting ten thousand ARIMA models every single night, and it is small enough that the number is not really the point — the point is that it is small enough to stop the conversation about whether you can afford to forecast nightly.

The same formula at other catalogue sizes, holding the four assumptions fixed:

     series    core-hours    wall clock on 32 cores    cost at $1.50/h
     10,000          3.33               6.3 min                  $0.16
    100,000         33.3                62.5 min                 $1.56
  1,000,000        333                  10.4 hours              $15.63

Only the third row is a problem, and it is a scheduling problem before it is a cost problem: 10.4 hours does not fit in an overnight window with any margin. The fix is more cores, not a different model class — 128 cores brings it to 2.6 hours at four times the hourly rate for a quarter of the time, so the dollar total is unchanged. Fitting independent models across series is embarrassingly parallel, and that is the single most useful fact about its cost.

The global neural run, costed

A global model inverts the shape of the bill: training is one large cost, inference across all series is almost free. Two more assumptions — a 25-minute training run on one GPU and 10,000 series inferred in 15 seconds once batched:

training      = (25 / 60) h × $2.00/h      = $0.83
inference     = (15 / 3600) h × $2.00/h    = $0.008

retrain nightly   = 365 × ($0.83 + $0.008)          ≈ $306 / year
retrain weekly    = 52 × $0.83 + 365 × $0.008       ≈ $46 / year

The interesting line is the last one. Because inference is three orders of magnitude cheaper than training, the retraining cadence — not the model size — sets the bill. Moving from nightly to weekly retraining cuts the cost by 85% while still producing a fresh forecast every night, and for most catalogues the parameters do not change meaningfully in a week. If you are paying for nightly retraining, check whether anything about the forecast changes because of it.

Note also that neither total is large. Both approaches cost less per year than the difference in engineering effort between them, which is the actual finding of this exercise and the reason the choice between them should be made on data volume and accuracy grounds, not on compute.

Why the mean fit time is the wrong number

The 6.3-minute wall clock above assumed every series takes 1.2 seconds. Automatic model selection does not behave like that: the time depends on how large a search the stepwise procedure runs before it converges, and the distribution is heavy-tailed. Suppose the mean is 1.2 s but the slowest 1% take 30 s each. The arithmetic changes:

slow tail    =    100 series × 30 s   =  3,000 core-seconds
rest         =  9,900 series × ~0.9 s =  9,000 core-seconds
total        =                          12,000 core-seconds  (mean still 1.2 s)

A quarter of the total work now sits in 1% of the series, and the wall clock is no longer 12,000 divided by 32. A parallel run finishes when its last task finishes, so the makespan can never be shorter than the single longest fit, and a scheduler that hands out work in large contiguous chunks will strand several slow series on one core while thirty-one others sit idle. Two fixes, both cheap:

• A per-series timeout with a fallback. Cap each fit at, say, five seconds and fall back to seasonal naive or simple exponential smoothing when it expires. The series that time out are usually the ones where the automatic search is thrashing because the data is degenerate, and a naive forecast on those is rarely worse than the one the search would have produced.
• Dynamic work distribution. Hand out series one at a time from a shared queue rather than splitting the catalogue into 32 equal slices up front. Equal slices are equal in series count, not in work.

What is actually expensive

Three things, none of them the model.

Memory, at the top of the range. A million series with three years of daily history is 1,000,000 × 1,095 ≈ 1.1 billion observations. In a long-format dataframe carrying a float64 value, an int64 timestamp and an int32 series id, that is roughly 1.1e9 × (8 + 8 + 4) ≈ 22 GB before indexes, before feature columns, and before whatever the library copies during a groupby. This is the constraint that actually breaks nightly runs, and it breaks them with an out-of-memory kill rather than a slow result. Chunk by series, or store the values as float32 and the id as a categorical and halve it.

Feature computation, if you have features. The costing above is for a model that takes only the series. Add lag features, rolling aggregates and calendar joins and the data preparation typically dominates the fitting — which is why the M5-style gradient-boosting pipelines are expensive to run despite the model itself training in minutes.

Everything a human touches. Fifty-seven dollars a year of compute against however you cost an engineer-day: the comparison is not close at any rate you might use. Retry logic, monitoring, the reconciliation step, the override workflow for planners and the investigation when a forecast looks wrong are all larger line items than the arithmetic on this page, and none of them scale down when you pick a cheaper model.