Why a Model's File Size Never Exactly Matches Its Parameter Count

Two bytes per parameter times eight billion parameters is 16 GB, and the file is 16.06 GB. Four bits per parameter times eight billion is 4 GB, and the file is 4.9 GB. The first gap is a rounding artefact. The second is 22% and has three separate causes, only one of which is the one people name.

The arithmetic everyone does first

The estimate is bytes-per-parameter multiplied by parameter count. It is a good estimate and you should keep using it — it is what VRAM planning rests on. But it is systematically low for quantized files, and knowing by how much and why stops you buying a 16 GB card for a model that needs 18.

The figures below come from llama.cpp’s llama-quantize README, which publishes measured bits-per-weight and sizes for Llama-3.1-8B at every supported quantization. Working backwards from its F16 row — 16.0005 bits per weight at 14.96 GiB — gives 14.96 × 2³⁰ × 8 ÷ 16.0005 ≈ 8.03 × 10⁹ parameters, which is the figure Meta publishes for that model. Every number below uses that same 8.03B.

A quantization block is not its name

“4-bit” does not mean four bits per weight. A k-quant stores weights in super-blocks, and each super-block carries scales and minimums alongside the packed quants. Those are real bytes and they are per-weight overhead.

The structures are in ggml’s ggml-common.h, where QK_K is 256. A block_q4_K holds two 16-bit super-block scales (4 bytes), a 12-byte packed scales-and-minimums array, and 128 bytes of 4-bit quants: 144 bytes for 256 weights.

Q4_K   144 bytes / 256 weights = 4.5   bits per weight
Q6_K   210 bytes / 256 weights = 6.5625 bits per weight
         (128 low bits + 64 high bits + 16 scales + 2 scale)
Q8_0    34 bytes /  32 weights = 8.5    bits per weight
         (32 quants + one 16-bit scale)

So the floor for a “4-bit” k-quant is 4.5 bits, not 4.0 — a 12.5% surcharge before anything else happens. On 8.03B parameters that alone is 8.03 × 10⁹ × 4.5 ÷ 8 = 4.52 GB rather than 4.02 GB.

A preset is a mixture, not a type

The second cause is larger than the first. Q4_K_M is not “every tensor as Q4_K”. llama-quantize deliberately holds some tensors at a higher precision, because the quality cost of squeezing them is out of proportion to the bytes saved — the attention value projections and feed-forward down projections in particular, plus the token embedding and output tensors, and it keeps one-dimensional tensors such as normalisation weights at F32 throughout. The README’s --pure flag exists precisely to switch that mixing off.

The mixture is why the README measures 4.8944 bits per weight for Q4_K_M rather than the 4.5 the block layout implies. Multiply out:

pure Q4_K   8.03e9 x 4.5000 / 8 = 4.52 GB   (block layout)
Q4_K_M      8.03e9 x 4.8944 / 8 = 4.91 GB   (measured mixture)
naive 4-bit 8.03e9 x 4.0000 / 8 = 4.02 GB   (what you guessed)

The measured line is where the README’s own memory-requirements table lands: it gives 4.9 GB for Llama-3.1-8B at Q4_K_M. The gap between the guess and the file is 0.89 GB, and 0.39 GB of that is the mixture while 0.50 GB is the block layout. Neither is metadata.

The same arithmetic scales. The README lists 43.1 GB for the 70B at Q4_K_M, against a naive 70 × 10⁹ × 4 ÷ 8 = 35 GB — and 70.6B × 4.8944 ÷ 8 = 43.2 GB, which is the published figure to within a rounding of the parameter count. If you are sizing hardware, use a measured bits-per-weight figure for the quantization you intend to run, not the digit in its name. Choosing that quantization is a separate decision covered in picking a quantization level.

What the metadata actually costs

The usual explanation for the gap is “tokenizer and metadata overhead”. It is a real cost and it is not the answer.

A GGUF file carries, in front of the tensors: a 24-byte header, the metadata key-value store, and a tensor-info table with a name, dimensions, type and offset for every tensor. The largest single item in the key-value store is the tokenizer — tokenizer.ggml.tokens is an array of every token string in the vocabulary, with tokenizer.ggml.merges and per-token scores or types alongside it. For a large vocabulary that runs to a few megabytes. Alignment padding, controlled by general.alignment and defaulting to 32 bytes, adds a negligible amount per tensor.

A few megabytes against 4.9 GB is under a tenth of a percent. It cannot explain a 22% gap and it will not move a hardware decision.

One reason this is easy to get wrong: llama.cpp’s reported model size is the sum of its tensor bytes, not the size of the file on disk, so the two are not the same quantity. If you want the non-tensor cost for a particular file, take the difference yourself — the file size from the filesystem, minus the tensor total that a header dump reports. That is a measurement one command long, and it is worth more than any figure quoted for somebody else’s file.

Safetensors checkpoints have the same shape of overhead in a different layout: an 8-byte little-endian length, then a JSON header giving every tensor’s dtype, shape and byte range, then the data. The header is small; a repository’s tokenizer files sit beside the weights as separate files rather than inside them, which is why a shard sum and a directory size disagree.

Gigabytes, gibibytes and rounded names

Two smaller effects account for most of the remaining confusion, and both are units rather than bytes.

• GiB is not GB. The same llama.cpp README calls the same Llama-3.1-8B Q4_K_M file 4.58 GiB in one table and 4.9 GB in another, and both are right: 4.58 × 1.073741824 = 4.92. A 7.4% discrepancy appears whenever one tool divides by 1024³ and another by 1000³. On a 70B this is over 3 GB of apparent difference in a file nobody has touched.
• The name is rounded. “8B” is 8.03B — 0.4% low. “70B” is nearer 70.6B. Model names round to something sayable, and on a large model the rounding is worth a gigabyte on its own.
• Shards do not sum the way you expect. A repository split into model-00001-of-00004.safetensors and friends may store tied input and output embeddings once while a parameter count counts them once too — or not, depending on the architecture. When a shard total disagrees with a model card by roughly the size of one embedding matrix, tied weights are the first thing to check.

Put together: start from a measured bits-per-weight figure, multiply by the real parameter count rather than the rounded one, divide by eight, and then decide whether the number you have is in GB or GiB before comparing it to a card’s capacity. The metadata you can ignore. And remember that the weights are only part of what has to fit — the KV cache grows with context and is charged separately.