Detecting Structural Breaks in a Time Series

A spike and a step look similar on a chart for a few observations and demand opposite responses: one should be down-weighted and forgotten, the other means everything before it should be discarded. The test that separates them is about parameters, not about points.

A break is not an outlier

The distinction is precise. An outlier is a single observation far from what the model predicted, with the model still correct — the parameters that generated the rest of the data also generated this point, and it landed in the tail. A structural break is a change in the parameters themselves: after some date, the relationship generating the series is a different relationship, and every subsequent observation comes from it.

The operational difference is persistence. An outlier does not change what happens next; a break changes everything that happens next. So the diagnostic question is not “how far is this point from the fit” but “does a model fitted before this date still describe the data after it”. That is what a Chow test asks.

It matters for forecasting in a very direct way. Almost every method in this cluster assumes the process is stable over the estimation sample. If it is not, a longer training window is actively harmful — the extra history is not weakly informative, it is from a different process — and the bias-variance argument for window width does not apply, because it assumed a drifting level rather than a changed one.

The Chow test, computed

Gregory Chow published the test in Econometrica in 1960, as “Tests of Equality Between Sets of Coefficients in Two Linear Regressions”. The logic is a comparison of fits. Fit the model once over the whole sample and record its residual sum of squares. Then fit it separately on each side of the candidate break date and add those two residual sums. If the parameters really are the same on both sides, splitting the sample buys almost nothing and the two totals will be close. If they differ, the split fits much better.

F  =   ( RSS_pooled − (RSS_1 + RSS_2) ) / k
      ─────────────────────────────────────────
           ( RSS_1 + RSS_2 ) / ( n1 + n2 − 2k )

k  = number of parameters in the model (including the intercept)
n1 = observations before the break, n2 = after

Work it on stated numbers. Take 24 monthly observations, a simple regression of demand on a time trend so k = 2 (intercept and slope), a candidate break after month 12, and these three residual sums: RSS_pooled = 480, RSS_1 = 120, RSS_2 = 140.

RSS_1 + RSS_2   = 120 + 140          = 260
improvement     = 480 − 260          = 220

numerator       = 220 / 2            = 110.0
denominator     = 260 / (24 − 4)     = 260 / 20 = 13.0

F               = 110.0 / 13.0       = 8.46      on (2, 20) degrees of freedom

The 5% critical value of an F distribution with 2 and 20 degrees of freedom is about 3.49, so 8.46 is well past it and the null hypothesis of equal coefficients is rejected. Splitting the sample explained 220 of the 480 pooled residual sum — 46% of it — which two extra parameters have no business achieving by chance.

One assumption in that F-statistic deserves attention because it is routinely violated: the test requires the error variance to be the same on both sides of the break. If what actually changed was volatility rather than the mean relationship, the denominator is not estimating a single quantity and the test can reject for the wrong reason. Check the two residual variances separately before you interpret the result. A variance-only change is a real and different finding, and it matters enormously for prediction intervals while leaving the point forecast alone.

Why an outlier lowers the statistic

Here is the part that makes the test genuinely useful for distinguishing the two cases, and it is rarely stated.

Suppose there is no break, just one wild observation in the second half. That observation inflates RSS_pooled — and it also inflates RSS_2, by very nearly the same amount, because splitting the sample does not help you fit a point that no line goes through. The numerator, which is the difference between them, barely moves. The denominator, which is the average residual in the split fits, goes up. So the ratio goes down.

A single outlier makes a Chow test less likely to fire, not more. A genuine level shift does the opposite: the pooled fit is forced to compromise between two levels and its residual sum is large, while each sub-sample fits its own level well, so the numerator is large and the denominator small. The test is naturally selective for exactly the thing you want it to detect, which is not true of most anomaly detectors — see anomaly detection for the methods aimed at the other case.

The practical confirmation, and it takes one line: refit with the suspect observation removed. If the break signal disappears, it was an outlier. If it strengthens, it was a break with an outlier sitting near it.

When you do not know the date

The Chow test needs a candidate date supplied from outside — a known price change, a system migration, a competitor opening. The obvious workaround, computing the statistic at every possible date and taking the largest, is invalid if you then use the ordinary F critical value. You have taken a maximum over many correlated tests, so the maximum is much larger than any single one would be under the null. Using 3.49 as the threshold after searching will find breaks in white noise.

Donald Andrews addressed this in Econometrica in 1993: the sup F statistic has a non-standard distribution with its own critical values, tabulated for the amount of the sample you searched over. The search must also be trimmed — conventionally excluding the first and last 15% of the sample — because a split very near an endpoint gives one sub-sample too few observations to estimate k parameters, and the statistic becomes unstable there rather than informative.

• Multiple breaks. Jushan Bai and Pierre Perron developed the framework for estimating several break dates at once and choosing how many with an information criterion, rather than testing for one break and stopping. This is the right tool when a series has been through several regimes.
• Sequential monitoring. Where you need to detect a break as it happens rather than after the fact, the CUSUM test of recursive residuals — Brown, Durbin and Evans, 1975 — accumulates one-step forecast errors and signals when the running sum leaves a boundary. It is designed for monitoring, which is a different statistical problem from testing a fixed sample.
• Non-regression changepoints. If you are looking for a shift in mean or variance rather than in regression coefficients, the changepoint literature — segmentation with a penalty on the number of changepoints, of which PELT is the standard efficient algorithm — is the better fit. See change point detection.

What to do once you have found one

Detection is the easy half. The three responses, in increasing order of how much data they preserve:

Truncate. Discard everything before the break and refit. Correct, simple, and expensive — if the break was recent you may have thrown away the seasonal history you needed, and a seasonal model on eight post-break observations is not going to work. Sometimes the honest answer is to fall back to a simpler model until enough post-break data accumulates.

Add an indicator. Include a dummy variable for the post-break period, and interact it with any coefficient you believe changed. This keeps the pre-break data contributing to the parameters that did not change, which is often most of them — the seasonal shape usually survives a level shift even when the level does not.

Down-weight rather than cut. Weight observations by age with a decay that steepens across the break, so pre-break data is used with reduced influence rather than discarded. This is the softest option and the hardest to justify to anyone, because the decay rate is a free parameter you chose.

Whichever you pick, re-examine the prediction intervals. A model refitted on a short post-break sample has large parameter uncertainty that its intervals will not reflect, so it reports confidence it has not earned — which, since a break is exactly when a forecast is most likely to be acted on, is the worst possible moment for it.