Skip to content
Multigrid

Quantization formats and what they cost

GPTQ, AWQ, GGUF's K-quants and the rest — what each one actually does to the weights, and where the quality goes when it goes.

Quantization is usually presented as a choice between file names. You are shown a list — Q4_K_M, GPTQ 4-bit, AWQ, NF4, 5.0bpw — with a size column and a vague ranking, and you pick the one that fits. That works until it does not: the model that fits produces subtly worse code, or the format your download is in has no kernel on your card, or the quantized weights are the same size as advertised and the process still runs out of memory. At that point the file name stops being enough and you need to know what the format actually did.

These pages take each format apart. What gets stored per weight, what gets stored per group, which tensors are treated differently from the rest, what the calibration data was for, and which step is the one that loses information. The arithmetic is done on the page from inputs you can check, because the interesting questions here — how much memory a group size costs, why 4-bit and not 3, why a vision tower is left alone — are answerable with division and a published constant, and answering them that way survives the next release of every tool involved.

GPTQ Explained: How Post-Training Calibration Works

The actual GPTQ loop — the layer-wise objective, the inverse-Hessian error update, and the three shortcuts that make it finish in hours instead of weeks.

10 min read

AWQ Explained: Protecting the Weights That Matter Most

How activation-aware quantization identifies salient channels from activation statistics and protects them with a scaling transform rather than higher precision.

9 min read

EXL2 and Fractional-Bit Quantization

How ExLlamaV2 mixes 2-, 3-, 4-, 5-, 6- and 8-bit quantization within one model to hit an arbitrary average bitrate, and what the measurement pass is measuring.

9 min read

bitsandbytes NF4 Quantization Explained

Why NormalFloat4 places its sixteen levels on the quantiles of a normal distribution instead of spacing them evenly, with the codebook values listed.

9 min read

Group Size in GPTQ and AWQ Quantization

What a group is, exactly how many bits per weight group sizes 32, 64 and 128 cost, and what changes in accuracy when you move between them.

8 min read

What a Calibration Dataset Actually Does During Quantization

Calibration data does not train anything — it decides which rounding errors are cheap, and the mechanism is different for GPTQ, AWQ and activation quantization.

9 min read

Quantization Granularity: Per-Tensor, Per-Channel and Per-Group Scales

One mechanism at three resolutions — how many weights share a scale — with the memory cost of each option worked out and the reason activations cannot use the fine ones.

9 min read

The Marlin Kernel and Why Quantized Weights Still Run Fast on GPU

Why a 4-bit matmul is a memory-bandwidth problem, why earlier W4A16 kernels lost their advantage as batch size grew, and what Marlin does about it.

9 min read

SmoothQuant Explained

How an offline per-channel rescaling moves quantization difficulty out of the activations and into the weights, making int8 activation quantization viable.

9 min read

Weight-Only Quantization Explained

Why W4A16 — quantized weights, full-precision activations — is the default for local inference, worked through the memory arithmetic and the batch-size limit.

9 min read

Dynamic Quantization for LLM Inference

What it means to compute a quantization scale at runtime instead of from calibration data, what that costs per token, and why weights never do it.

8 min read

What K, M and S Mean in a GGUF Quant Name

Decoding Q4_K_M and its relatives tensor by tensor — what the K structure is, and which specific tensors the S, M and L variants promote.

9 min read

Why 4-Bit Quantization Became the Local-Inference Default

An argument that 4 bits won on three independent constraints — the size curve, the published quality cliff below it, and byte alignment — none of which is about quality alone.

10 min read

Outlier Features and Why LLM Quantization Breaks Without Handling Them

The emergent outlier phenomenon in transformer activations, the measured evidence for it, and why it kills naive int8 but leaves weight-only 4-bit alone.

9 min read

How Long GPTQ Calibration Takes and What It Needs in Memory

The one published wall-clock anchor for GPTQ, a peak-VRAM model derived from layer shapes, and why quantizing a 70B does not need a 70B's worth of memory.

10 min read

INT4 Accuracy Loss by Model Size, Derived

The published per-size perplexity deltas for 4-bit quantization, a mechanism for why they shrink with scale, and an honest account of what the table does not show.

10 min read

Mixed-Precision Quantization: Keeping Some Layers at Higher Bits

How sensitivity ranking decides which layers get more bits, with the memory arithmetic for a non-uniform bake against a uniform one.

9 min read

Quantizing a Vision-Language Model: What's Different

Why the vision tower and projector are excluded from quantization, why the calibration set has to contain images, and what breaks when it does not.

9 min read

Other topics