Time Series Foundation Models: What Chronos and TimeGPT Actually Do

A forecasting foundation model is trained once on a large collection of unrelated series and then asked for a forecast on a series it has never seen, with no fitting step. That only works if something about a series is transferable between domains. Knowing exactly what that something is tells you when to expect it to work.

What actually transfers

Nothing about the units transfers. A model pretrained on electricity load, web traffic and river levels has no notion of megawatts, and if it did, the notion would be useless for your series. What transfers is shape: the way a level drifts, the way a periodic component sits on top of it, the way variance grows with level, the way a spike decays. Those are properties of a scaled series, and every design in this family removes scale before the model sees anything.

Chronos, from Amazon Science, does this with mean scaling: it divides the context window by the mean absolute value of that window, so a series measured in thousands and a series measured in fractions arrive at the model looking like the same object. The paper by Ansari and colleagues, published in 2024, states the transform as subtracting a mean of zero and dividing by the mean of the absolute values over the historical context. That one step is what makes a single set of weights usable across domains, and it is also the first thing to check when a zero-shot forecast comes back at an implausible level.

The second thing that transfers is periodicity relative to the sampling rate. A model that has seen a great deal of hourly data has seen period-24 and period-168 structure repeatedly, in many domains. It has learned that hourly series often behave that way. It has not learned that your hourly series does, and it will find out only from the context you hand it.

Design one: turn numbers into a vocabulary

The first design treats forecasting as language modelling literally. After scaling, Chronos quantises the real line into a fixed set of bins, each with a centre, and replaces every observation with the index of its bin. The vocabulary is 4,096 entries: the quantisation bins plus a padding token, which also stands in for missing values, and an end-of-sequence token. The result is a sequence of integers, and a T5-family encoder-decoder is trained on it with ordinary categorical cross-entropy — what the paper calls regression via classification. The released sizes run from about 20M parameters up to 710M.

Two consequences follow directly. First, the model’s output is a distribution over bins, so a probabilistic forecast is free: you sample a token, append it, sample again, and repeat for the horizon, exactly as text generation works. Run that several times and you have a set of sample paths from which quantiles are read off. Second, the loss does not know that bin 300 is close to bin 301. Cross-entropy over an unordered vocabulary charges the same price for a near miss as for a wild one, and any ordering the model respects is learned rather than imposed.

Quantisation also puts a floor and a ceiling on what can be expressed. A value that falls outside the outermost bins after scaling is clamped to the outermost bin, and resolution inside the range is limited by the bin width. For a series that spends most of its life in a narrow band and occasionally jumps by an order of magnitude, that is a real constraint rather than a rounding detail.

Design two: patches and quantile heads

The second design drops the vocabulary. Instead of one token per observation, the context is chunked into patches of several consecutive observations, each patch becomes one input embedding, and the decoder emits the whole horizon at once with heads that predict quantiles directly. Amazon’s own Chronos-Bolt works this way, and the project’s repository describes it as direct multi-step forecasting producing quantile forecasts, in contrast to the original Chronos, which samples trajectories. The repository claims Bolt is up to 250 times faster and 20 times more memory-efficient than the original at the same size.

Patching is not a cosmetic change. A patch of size 16 makes the sequence the transformer attends over sixteen times shorter, and attention cost grows with the square of sequence length, so the same hardware can hold a much longer context. Direct multi-step output also removes autoregressive error accumulation over the horizon, at the cost of no longer producing coherent sample paths: quantile heads give you a marginal interval at each future step, not a joint trajectory you can sum across the horizon. If you need the distribution of a total over the next fourteen days, that distinction decides which design you can use.

TimeGPT, from Nixtla, is a third point in this space and is the one you reach through a hosted API rather than a checkpoint. The 2023 paper by Garza, Challu and Mergenthaler-Canseco describes an encoder-decoder transformer with residual connections and layer normalisation, local positional encoding, and a linear layer mapping the decoder output to the forecast window. It reports training on over 100 billion data points drawn from domains including finance, weather, energy, web traffic and transport, and it produces prediction intervals by conformal calibration — rolling forecasts on the recent history of your own series to estimate the model’s error on it.

What is in the corpus, and what is not

Real forecasting corpora are much smaller and much less diverse than text corpora, which is why the Chronos authors generated synthetic series to fill the gap. Their KernelSynth procedure samples Gaussian process kernels from a bank containing linear kernels for trend, radial-basis kernels for smooth variation and periodic kernels for seasonality, composes several of them with addition and multiplication, and draws series from the resulting prior. That is a deliberate statement about what the model should consider a plausible series: sums and products of trend, smoothness and periodicity.

It is also a statement about what is absent. A synthetic corpus built from those primitives contains no promotions, no stockouts, no public holidays and no regime changes caused by a pricing decision. Nothing in pretraining teaches a model that the fourth Thursday in November is different, and the original Chronos is univariate, so there is no channel through which to tell it at inference time either. If your series is driven by things you know in advance, see forecasting with exogenous variables, because that is the capability you are giving up.

Where the idea stops

• The context window is the memory. The original Chronos was trained with a context of 512 tokens and a prediction length of 64. One token is one observation, so 512 observations is all the history the model can look at. Daily data with an annual cycle needs 730 observations to show two complete cycles, which does not fit.
• A hierarchy is not preserved. Forecast every store separately and the store forecasts will not add up to the chain forecast. Nothing in a univariate foundation model addresses that; see hierarchical forecast reconciliation.
• Short and intermittent series are the hard case. A series of mostly zeros with occasional demand has almost no shape to transfer, and the scaling step divides by a mean absolute value that is dominated by the zeros.
• Published scores are not your score. Every evaluation in this literature is an average over a benchmark collection. The useful number is the one you get by holding out the last h periods of your own series and comparing against a seasonal naive baseline, which is free and is beaten less often than people expect.