Long-Horizon Forecasting: Why Error Compounds Past a Few Steps

“Error compounds” is true and useless on its own. The useful version is that forecast variance at horizon h is a weighted sum of h future shocks, the weights are the model’s impulse response, and whether the sum converges decides whether your error curve flattens or keeps climbing.

Where the growth actually comes from

Write any linear time series model in its moving-average form: the value at time t is a weighted sum of the shocks that have hit it, with weights ψ₀, ψ₁, ψ₂, … called the impulse response — ψ_j is how much of a shock j periods ago is still present today, and ψ₀ = 1 by construction.

When you forecast h steps ahead, you know every shock up to time t and none of the h shocks that arrive after it. Those unknown shocks are exactly the forecast error, so if each shock has variance σ² and they are independent:

Var( error at horizon h ) = σ² · ( ψ₀² + ψ₁² + … + ψ_{h−1}² )

That is the whole result, and it is the standard expression for the forecast distribution of an ARIMA model. Hyndman and Athanasopoulos derive it for each model class in Forecasting: Principles and Practice, which is free online and is the reference to check against. Note what it says: the error at horizon h is not the one-step error applied h times. It is a sum of h terms, and the terms shrink or they do not.

The unbounded case: error grows as √h

For a random walk — tomorrow is today plus a shock — every shock is permanent, so ψ_j = 1 for all j. The sum of h ones is h, and:

Var(h) = h · σ²        RMSE(h) = σ · √h

Take a stated one-step root mean squared error of σ = 100 units — the assumption, and everything below is arithmetic over it:

h =  1     RMSE = 100 · √1  = 100
h =  4     RMSE = 100 · √4  = 200
h = 12     RMSE = 100 · √12 = 346
h = 24     RMSE = 100 · √24 = 490
h = 52     RMSE = 100 · √52 = 721

Two things fall out. Error grows without limit, so there is no horizon at which a random-walk forecast becomes reliable — but it grows sublinearly. Twenty-four steps out the error is 4.9 times the one-step error, not 24 times. People who describe compounding as multiplicative overstate it badly for this model class; people who expect the error to settle down overstate the other way.

The 95% interval widens the same way, as ±1.96 · σ · √h, which is the familiar funnel shape on a chart. If your intervals are drawn flat across the horizon, either the model believes the series is stationary or the intervals are not coming from the model at all.

The bounded case: error stops growing

Now take a stationary AR(1): each period retains a fraction φ of the last, with |φ| < 1. The impulse response decays geometrically, ψ_j = φ^j, and the sum is a geometric series:

Var(h) = σ² · (1 − φ^{2h}) / (1 − φ²)   →   σ² / (1 − φ²)  as h → ∞

With φ = 0.8 and the same σ = 100, the denominator is 1 − 0.64 = 0.36 and the limit is 100/√0.36 = 167:

h =  1     RMSE = 100 · √1.000 = 100
h =  2     RMSE = 100 · √1.640 = 128
h =  4     RMSE = 100 · √2.312 = 152
h =  8     RMSE = 100 · √2.700 = 164
h = 12     RMSE = 100 · √2.778 = 167
h → ∞      RMSE = 100 / √0.36  = 167

By horizon 12 the error has reached its ceiling to three figures and stops. It stops because the forecast itself has converged to the unconditional mean, and the error of forecasting the mean is just the series’ own standard deviation. Beyond that point the model has nothing left to say and is not getting any worse at saying it — which is a genuinely different failure from the random walk, and the diagnosis is worth having. A long-horizon forecast that is flat and whose intervals have stopped widening is not broken; it is a stationary model telling you it has run out of information.

Seasonal models add a wrinkle worth knowing about: their impulse response is periodic, so the error curve is not monotone. For a seasonal naive forecast on monthly data, the horizon-12 forecast uses the same-month observation from one year back, while horizon 11 uses a value from a different month of a different year, so error at 12 can be lower than at 11. Any summary that reports “average accuracy over the horizon” flattens that structure away. Plot error against h rather than averaging it, and the periodicity is visible immediately — and if it is not, your model is not using the seasonality you think it is.

Most real business series sit between these two. A trend-stationary series with seasonality has an impulse response that decays but not to zero within the horizon you care about, so the curve is somewhere between √h and flat. Fit the model, extract its ψ weights, and you have the curve for your series rather than a rule of thumb.

The error the formula leaves out

The expression above assumes the model is correct and its parameters are known. Neither holds, and both extra sources of error behave worse at a long horizon than the shock term does.

• Parameter uncertainty. You estimated φ from finite data. An error in φ is raised to the power h in the forecast, so a small estimation error is amplified geometrically as the horizon grows. Most software’s prediction intervals ignore this entirely and are therefore too narrow far out, in a direction the software does not warn you about.
• Recursive versus direct strategies. A recursive multi-step forecast feeds its own prediction back in as an input, so model misspecification compounds on top of the shock term. A direct strategy trains a separate model per horizon, which does not compound misspecification but sees fewer effective training targets per model and cannot share structure across horizons. Neither dominates; the recursive one degrades faster when the model is wrong, which is most of the time.
• Exogenous inputs need forecasts of their own. A model that uses price or weather as a driver needs those values at every future step. Beyond the horizon at which they are actually known, you are stacking one forecast on another and the combined error is not in any interval your model produced. See forecasting with exogenous variables.

What to do at a long horizon

Forecast a distribution rather than a number. Once the interval is three times the one-step error, the point forecast is a summary of something very wide and any decision made on it alone is being made without its most important input; quantile regression gives you the range directly rather than assuming a Gaussian shape around the mean.

Aggregate to a coarser granularity when the decision allows it. A weekly total 12 weeks out has far less relative error than a daily figure 84 days out, for the reason worked through in choosing a forecast granularity. And score long-horizon accuracy separately from short: an average over all horizons hides which end of the range is failing, and the fixes for the two ends have nothing in common.