Intermittent Demand: Why Standard Forecasting Models Fail on Sparse Series

On a series that is zero most weeks, the model that always predicts zero wins on mean absolute error and loses on mean squared error, and neither result has anything to do with which forecast is useful. The metric decides the answer before any model is fitted, and that is the first thing to fix.

Twenty periods, four sales

Here is the series everything below is computed on. Twenty weeks of demand for a slow-moving spare part:

week    1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20
demand  0  0  5  0  0  0  0  3  0  0  8  0  0  0  0  0  4  0  0  0

total demand = 20 units over 20 weeks   →  mean rate = 1.0 units/week
non-zero periods = 4                    →  16 of 20 weeks are zero

Two candidate forecasts, both constant. Forecast A says 0 every week. Forecast B says 1.0 every week, the arithmetic mean.

The metric picks the model

MAE, forecast A (0):   (16×0 + 5 + 3 + 8 + 4) / 20        = 20 / 20 = 1.00
MAE, forecast B (1):   (16×1 + 4 + 2 + 7 + 3) / 20        = 32 / 20 = 1.60

MSE, forecast A (0):   (16×0 + 25 + 9 + 64 + 16) / 20     = 114 / 20 = 5.70
MSE, forecast B (1):   (16×1 + 16 + 4 + 49 + 9) / 20      =  94 / 20 = 4.70

MAPE, either:          16 divisions by zero               = undefined

Mean absolute error prefers the forecast that is never right by 60%. Mean squared error prefers the mean by 18%. Mean absolute percentage error cannot be computed at all. These are not close calls and they point in opposite directions.

The reason is not a quirk of this example, it is the definition of the two losses. Absolute error is minimised by the median of the distribution, and on a series where more than half the periods are zero the median is zero. Squared error is minimised by the mean, which is 1.0. Whenever a series is more than 50% zeros, any model trained to minimise absolute error has a globally optimal solution of “always predict zero”, and gradient descent will find it. This is not the model failing; it is the model succeeding at the objective it was given.

Percentage errors fail for the more obvious reason and one less obvious one: even where demand is non-zero, the denominators are tiny, so a one-unit miss on a week that sold 1 is a 100% error, and the aggregate is dominated by the smallest observations. The scaled errors — MASE, and the RMSSE used to score the M5 competition — exist for exactly this: they divide by the in-sample error of a naive forecast rather than by the observation, so a zero in the data does not produce a division by zero.

Three consequences for how you set up the problem. Never score intermittent series with a percentage error. Do not compare a point forecast against a target that is zero four times in five and expect the comparison to mean anything. And decide what the forecast is for before choosing a loss, because the answer is usually a stocking decision, and a stocking decision needs a quantile rather than either the mean or the median.

What Croston’s method does

J. D. Croston’s 1972 method is the standard parametric approach and it changes the problem rather than the model. Instead of smoothing the demand series, it splits it into two series and smooths each separately:

• Demand size z — the values in the non-zero periods only: 5, 3, 8, 4.
• Inter-demand interval p — the number of periods between consecutive non-zero periods.

Both are updated with simple exponential smoothing, only in periods where demand occurred, and the per-period forecast is the ratio:

forecast per period  =  ẑ / p̂

on the series above, at convergence:
    ẑ ≈ mean non-zero size     = 20 / 4 = 5.0 units
    p̂ ≈ mean interval          = 20 / 4 = 5.0 periods
    forecast                    = 5.0 / 5.0 = 1.0 units per period

The same 1.0 the naive mean gave, which looks like it bought nothing. It bought two things. The two components are separately interpretable — you now know the answer is “5 units every 5 weeks” and not “1 unit a week”, which is a completely different stocking problem. And the two smoothing constants can differ, so the size can adapt quickly while the interval stays stable, or the reverse.

Croston’s estimator is biased upward. Aris Syntetos and John Boylan showed that taking the ratio of two independently smoothed quantities over-forecasts mean demand, and proposed the Syntetos-Boylan approximation: multiply the Croston forecast by a deflating factor that is linear in the smoothing constant used for the intervals. With an interval smoothing constant of 0.1, the correction scales the forecast by 0.95 — small, systematic, and in the same direction on every SKU, which is exactly the kind of bias that accumulates into visible over-stocking across a catalogue.

The failure Croston cannot see

Look again at the update rule: both components are updated only when demand occurs. Now consider a part that stops selling entirely. Week 20 was a zero, so was week 21, so is every week after. Croston’s estimates never update again, and the forecast sits at 1.0 units per week forever while actual demand is zero forever. The method has no mechanism for decaying towards zero, because its clock only ticks on demand.

This is the obsolescence problem, and it is severe in practice: a spare parts catalogue accumulates dead SKUs continuously, and a forecaster that cannot notice one is a forecaster that keeps recommending stock for parts nobody will ever order again.

Ruud Teunter, Aris Syntetos and Mohamed Zied Babai proposed the fix in 2011. The TSB method replaces the inter-demand interval with a demand probability, and updates that probability every period — downwards in a zero period, upwards in a non-zero one. A part that stops selling has its probability smoothed steadily towards zero, and the forecast decays with it. If you are choosing one intermittent method for a catalogue with any turnover in it, this is the property that matters more than the accuracy margin.

For deciding which method suits which SKU, the classification scheme of Syntetos, Boylan and Croston splits the catalogue on two statistics: the average inter-demand interval, and the squared coefficient of variation of the non-zero demand sizes. Series with long intervals and highly variable sizes — the “lumpy” quadrant — are the ones where no point forecast is going to be satisfactory, and that is a useful thing to know before you try to improve one.

Forecast a distribution instead

The decision an intermittent forecast feeds is almost always an inventory one: how many to hold so that the probability of a stockout over the lead time is below some target. That question needs P(demand over lead time ≥ x), and no point forecast contains it.

The natural model is compound: a Bernoulli process for whether demand occurs and a separate distribution for how much when it does. From those two you can construct the lead-time demand distribution and read the service-level quantile off it directly. Croston and TSB give you the two components already — they just throw the structure away when they take the ratio.

Where you are fitting a model rather than deriving one, quantile regression gives the same output shape and handles zeros without complaint, because the pinball loss has no denominator. Two cautions specific to sparse data: a quantile below the zero proportion is exactly zero and carries no information, so on a series that is 80% zeros there is nothing to learn below the 80th percentile; and quantiles are marginal per period, so you cannot add the 95th percentile across a four-week lead time to get the 95th percentile of the four-week total. Simulate the path, or model the lead-time aggregate directly at a coarser granularity where the series is no longer intermittent — which is very often the simplest correct answer available.