How Long GPTQ Calibration Takes and What It Needs in Memory

There is exactly one authoritative wall-clock figure for GPTQ and it is from 2023. Everything else circulating is somebody’s run on hardware you do not have. What generalises is the memory model, which is derivable from layer shapes, and the shape of the time curve, which is derivable from operation counts — so this page derives both and says plainly where the derivation stops predicting reality.

The one published anchor

The GPTQ paper (Frantar, Ashkboos, Hoefler and Alistarh, ICLR 2023) reports its own runtimes for the two largest models it quantized:

• OPT-175B in 4.2 hours.
• BLOOM-176B in 3.8 hours.
• Both on a single NVIDIA A100 with 80 GB.
• Calibration was 128 random 2048-token segments of C4, about 262,000 tokens in total.

Read that against the alternative it was competing with, and the headline is not the number but the order of magnitude: a 175-billion parameter model, on one GPU, in an afternoon. The paper is the primary source and is worth reading for the method rather than the timing.

Peak VRAM, derived from layer shapes

The memory model is the part you can actually compute, and it is more useful than the timing because it decides whether a run is possible at all. GPTQ quantizes one transformer block at a time. The full model never needs to be resident on the GPU — only the current block, its Hessians, and the calibration activations flowing into it. Peak VRAM is therefore approximately:

peak ≈ block_weights
     + largest_hessian
     + calibration_activations
     + workspace

block_weights           = params_per_block * 2 bytes        (fp16)
largest_hessian         = max(d_in)^2 * 4 bytes             (fp32)
calibration_activations = 2 * n_samples * seq_len * hidden * 2 bytes
                          (input and output buffers)

Every input to that is a number you can read off a model config: num_hidden_layers, hidden_size, intermediate_size, num_key_value_heads, plus the two you chose — n_samples and seq_len. The Hessian term is the non-obvious one: it is the second-moment matrix over input channels, so it is square in the layer’s input dimension and completely independent of how much calibration data you use.

Two worked examples

Both use 128 samples of 2048 tokens, matching the paper. All shapes are from the models’ published configurations; all arithmetic is on this page.

An 8B-class model — 32 layers, hidden 4096, intermediate 14336, grouped-query attention with 8 KV heads:

params per block
  q 4096x4096 = 16.8M   o 4096x4096  = 16.8M
  k 1024x4096 =  4.2M   v 1024x4096  =  4.2M
  gate/up/down 14336x4096 x3 = 176.2M
  total                       = 218.2M  ->  0.44 GB in fp16

largest Hessian  (down_proj, d_in = 14336)
  14336^2 * 4 bytes           = 0.82 GB

calibration activations
  2 * 128 * 2048 * 4096 * 2   = 4.29 GB

peak (before workspace)       ≈ 5.6 GB

A 70B-class model — 80 layers, hidden 8192, intermediate 28672, 8 KV heads:

params per block
  q,o 8192x8192 x2            = 134.2M
  k,v 1024x8192 x2            =  16.8M
  gate/up/down 28672x8192 x3  = 704.6M
  total                       = 855.6M  ->  1.71 GB in fp16

largest Hessian  (down_proj, d_in = 28672)
  28672^2 * 4 bytes           = 3.29 GB

calibration activations
  2 * 128 * 2048 * 8192 * 2   = 8.59 GB

peak (before workspace)       ≈ 13.6 GB

That second result is the one worth taking away. Quantizing a 70B does not need 70B-scale memory; on this model it needs somewhere in the region of a large consumer card, because only one of eighty blocks is ever resident. The FP16 checkpoint still has to be read from disk and the untouched blocks held somewhere — system RAM or disk, which is what the tooling’s CPU-offload options are for — but the GPU requirement is set by one block, not the model.

Two levers follow directly from the formula. Halving n_samples or seq_len halves the largest term in the 70B case, at the cost of a worse-conditioned Hessian; see what a calibration dataset actually does for what that costs. The Hessian term cannot be reduced this way at all — it is fixed by intermediate_size — which is why models with very wide MLPs are harder to quantize on a small card than their parameter count suggests.

Where the time goes, and why FLOPs mispredict it

The operation count is computable and instructive precisely because it does not reproduce the published wall clock. Three components, for a model of P quantizable parameters and N calibration tokens:

1. forward passes to produce activations
     ≈ 2 * P * N  FLOPs

2. Hessian accumulation, per linear layer
     ≈ 2 * d_in^2 * N  FLOPs

3. the GPTQ column loop, per matrix
     ≈ 2 * d_out * d_in^2  FLOPs

Evaluate for the 70B shapes above with N = 262,144 tokens: term 1 is about 3.7e16 FLOPs, term 2 about 4.3e16, term 3 about 1.9e15 — call it 8.2e16 FLOPs in total. An A100 sustaining an optimistic 100 TFLOP/s on that mix would finish in roughly 14 minutes.

The paper reports 4.2 hours for a model more than twice that size. The arithmetic is off by well over an order of magnitude, and the gap is the finding, not an error. GPTQ’s inner loop is dominated by reading and writing large matrices rather than by multiplying them — the Cholesky decomposition and triangular solves run at a small fraction of peak, the calibration activations are shuttled between host and device when offloading is enabled, and the checkpoint is streamed from disk block by block. None of that is in a FLOP count.

What the operation count is good for is scaling. All three terms are linear in N, so doubling the calibration set roughly doubles the run. Terms 1 and 3 are linear in P. Term 2 and 3 are quadratic in d_in, which is why a wide model costs more per parameter than a deep one. Those relationships hold regardless of what your hardware achieves in absolute terms.

Measuring it on your own machine

Since no published figure will predict your run, measure the cheap version and extrapolate. The work is per-block and nearly identical across blocks, so one block tells you almost everything.

1. Read the four numbers off the model’s config.json: num_hidden_layers, hidden_size, intermediate_size, num_key_value_heads. Put them through the VRAM formula above before starting anything.
2. Start a run with your intended n_samples and seq_len and watch peak memory with nvidia-smi --query-gpu=memory.used --format=csv -l 1, or through whatever GPU monitoring you already run.
3. Time the first two or three blocks and multiply by num_hidden_layers. Skip the first block in the average — it includes one-off setup and the initial forward pass over the whole calibration set.
4. If peak memory is the problem, cut n_samples before you cut seq_len: sequence length is what makes the activation statistics representative of long-context deployment, and it is the more expensive thing to lose.
5. If it still does not fit, enable the tooling’s CPU-offload path. The trade is wall-clock time for VRAM, and given that the run is already dominated by data movement rather than arithmetic, the penalty is real.