What a Calibration Dataset Actually Does During Quantization

Every calibrated quantization method asks you for text, and almost every guide tells you to use the default and move on. The text is not teaching the model anything. It is deciding, weight by weight, which rounding errors are affordable — and once you can see how, the question of which corpus to use answers itself.

It is not training data

No gradient is computed. No parameter is updated in the direction of a loss. The calibration corpus is run forward through the model exactly once per layer, and what is kept is a statistic about the activations that arrive at each linear layer — not about the model’s predictions, not about whether they were any good.

That distinction has a practical consequence people get wrong constantly: calibrating on your own domain data does not make the model better at your domain. It makes the model’s quantization error smaller on inputs that look like your domain, which is a much weaker and much more specific claim. A quantized model calibrated on Python will not have learned Python; it will have rounded its weights in a way that costs less when Python-shaped activations flow through it.

Three methods, three different uses

“Calibration data” names the input, not the mechanism. Three families of method consume it in genuinely different ways, and the differences explain why they respond differently to a bad corpus.

• GPTQ builds a Hessian from it. The statistic is XXᵀ, the second-moment matrix of the activations arriving at the layer. It is used to decide how much the remaining unquantized weights should be nudged to compensate for each rounding error, and it is a full matrix over input channels — so it captures not only which channels are large but which are correlated. This is the richest use of the data and also the one most able to overfit it.
• AWQ takes per-channel magnitudes from it. The statistic is a vector: the mean absolute activation per input channel. It is used to pick the per-channel scaling that protects salient channels, and one exponent per layer is grid-searched against it. Far less capacity to overfit, which the AWQ paper gives as an explicit design motivation.
• Static activation quantization takes ranges from it. When activations themselves are being quantized with fixed scales — the W8A8 static case — the calibration set determines the numeric range each activation tensor is assumed to occupy at runtime. This is the use with the sharpest failure mode: a real input that exceeds the calibrated range clips, and clipping an outlier feature damages the output badly. Dynamic quantization avoids this by computing the range at runtime instead.

Weight-only methods are therefore fairly forgiving of an imperfect corpus and static-activation methods are not, which is a large part of why weight-only is the default for local inference.

How a different corpus changes the rounding

Concretely, in the GPTQ loop. The error compensation applied after rounding column j is scaled by the Hessian’s entries for that column, so a column whose input channel is heavily exercised by the calibration text gets a large, carefully computed correction, and a column whose channel is nearly dormant in that text gets almost none.

Now suppose the corpus is entirely English prose and the model will be asked to write code. Channels that carry the structure of indentation, bracket matching and identifier tokens are lightly exercised, so the Hessian says they are cheap to damage, so the algorithm spends its compensation budget elsewhere. Nothing has gone wrong from the algorithm’s point of view — it minimised the objective it was given, on the distribution it was given. The distribution was simply not the one you deploy on.

The same argument runs on chat templates. A base-model corpus contains none of the special tokens an instruction-tuned model wraps every turn in, so the channels those tokens activate are invisible to the calibration pass. This is the single most common practical calibration mistake: quantizing an instruct model on raw web text and then finding it drifts out of its own response format.

Choosing the corpus

1. Start from what the model will see. If it will read source files, include source files. If every request is wrapped in a chat template, apply the template to the calibration samples too, special tokens and all.
2. Keep it diverse enough to cover the channels. A corpus of a hundred near-identical prompts exercises a narrow slice of the activation space and leaves the rest of the model rounded on essentially no information. Mixing in a general corpus alongside your domain text is the standard hedge.
3. Match the sequence length to deployment. Activation statistics from 256-token samples say little about what a 16k-token request looks like at the layers where long-range structure shows up.
4. Use the published defaults when you have no better information. The GPTQ paper used 128 random 2048-token segments of C4, and GPTQConfig still accepts "c4", "c4-new" and "wikitext2" by name. A generic corpus is a defensible default; a badly-chosen specific one is worse than either.
5. Evaluate the result on your task, not on the calibration set. Reconstruction error against the data you calibrated on is the objective the algorithm just minimised, so it is guaranteed to look good and tells you nothing.

How much, and how long

The corpus is small by any training standard and that is not an accident. The objective is a per-layer least-squares reconstruction over input channels, so what the data has to do is make XXᵀ well-conditioned — that is, provide enough independent samples to estimate a matrix whose side length is the layer’s input dimension. A few hundred thousand tokens does that. Adding millions more does not improve the estimate proportionally, and it multiplies the runtime directly.

The GPTQ paper’s configuration — 128 samples of 2048 tokens, about 262,000 tokens in total — remains the reference point, and most tools default to something within a factor of two of it. Where the cost of increasing it actually lands, and what dominates peak memory during the run, is worked through in how long GPTQ calibration takes.