GGUF Filename Conventions on Hugging Face, Decoded

A quantization repository will hand you twenty files whose names differ by six characters. Most of those characters are specified, a few are one uploader’s habit, and telling the two apart is the whole skill.

The format string

The ggml project publishes a naming convention in its GGUF specification document, and it is a single pattern with hyphens between the parts that are present:

[<Sidecar>]<BaseName><SizeLabel><FineTune><Version><Encoding><Type><Shard>.gguf

Only three of those are required for a name to be considered conformant: BaseName, SizeLabel and Version. Everything else is optional, which is why real filenames vary so much in length while still being legal. The specification is explicit that the convention exists so a human can read the important details at a glance, and that it is not intended to be reliably machine-parseable, because the corpus of GGUF files already in the world predates it.

Take a file from a typical quantization repository: Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf. BaseName is Meta-Llama-3.1, SizeLabel is 8B, FineTune is Instruct, Version is absent so it defaults to v1.0, and Encoding is Q4_K_M. There is no Type field, which is itself the signal: Type is omitted for ordinary tensor files and present only for things that are not one.

Field by field

• BaseName — the architecture or model family, taken from the general.basename metadata key with spaces turned into dashes. This is the part that tells you what the weights are, and it is the part uploaders edit most freely.
• SizeLabel — written as <expertCount>x<count><scale-prefix>, where the prefix is K, M, B, T or Q for thousand through quadrillion. So 8B is eight billion parameters and 8x7B is a mixture-of-experts model with eight experts of seven billion. One decimal point is allowed, which is how 1.5B is legal.
• FineTune — the optimisation goal, from general.finetune: Instruct, Chat, Coder and so on. A base model omits it, and that omission is worth noticing before you wonder why the thing will not follow instructions.
• Version — v<Major>.<Minor>, defaulting to v1.0 when absent. In practice most uploaders fold the version into BaseName (Meta-Llama-3.1) rather than using this field.
• Encoding — the quantization scheme: F16, Q8_0, Q4_K_M, IQ3_XS. This is the field you are usually comparing across, and the difference between two adjacent encodings is smaller than the number of files suggests.
• Type — LoRA for an adapter, vocab for a metadata-and-tokenizer-only file, absent for a normal model. A file with LoRA in this position is not something you can load on its own.

Shards and sidecars

A model too large for one file is split, and the split is written as <ShardNum>-of-<ShardTotal> with both numbers zero-padded to five digits and counting from 00001. The specification’s own example is Grok-100B-v1.0-Q4_0-00003-of-00009.gguf. You need every shard; the runtime opens the first one and finds the rest by name, so renaming a single shard breaks the set.

Sidecars sit at the very front of the name and mark a file as an auxiliary module rather than a standalone model. The two currently documented are mmproj, a multimodal projector that turns image or audio embeddings into something the language model can consume, and mtp, multi-token-prediction heads used for speculative decoding. So mmproj-Qwen2-VL-7B-v1.0-F16.gguf is not a model you run — it is the vision half, loaded alongside the text half, and downloading it on its own is a common first mistake with vision repositories.

The suffixes nobody standardised

Now the part that causes the confusion. Browse any large quantization repository and you will find names the specification does not describe. In the Hugging Face repository bartowski/gemma-2-9b-it-GGUF, for instance, files include gemma-2-9b-it-Q6_K_L.gguf, gemma-2-9b-it-Q4_K_M-fp16.gguf and gemma-2-9b-it-Q6_K-f32.gguf alongside the plain Q4_K_M and Q6_K.

None of those trailing letters are in the ggml spec. They are one uploader’s notation for a real thing: an _L or _XL suffix generally means the embedding and output tensors were left at a higher precision than the body of the model, and a -fp16 or -f32 suffix means some tensor was kept unquantized. That is why gemma-2-9b-it-Q4_K_M-fp16.gguf is 6.37 GiB against 5.37 GiB for the plain Q4_K_M in the same repository — an extra gigabyte, all of it in tensors the base quant would have compressed.

The important consequence: you cannot compare an _L file from one uploader with an _L file from another and assume they mean the same thing. The suffix is a claim about which tensors were spared, and only the uploader’s own model card defines it. Some repositories also prefix quants with i1- to mark that an importance matrix was used during quantization, which is again a convention rather than a field.

What the name does not tell you

Three things you will want, that the filename cannot give you. First, which build of llama.cpp produced it — the per-tensor mixtures behind Q4_K_M live in source and have been edited more than once, so two files with identical names quantized a year apart are not byte-identical and need not be the same size. Second, whether an importance matrix was used, unless the uploader added a marker. Third, the actual file size, which is the number that decides whether it fits.

All three are answerable without downloading anything. The repository file listing gives you exact byte sizes, and the GGUF header itself carries general.file_type, general.quantization_version and whatever metadata the quantizer wrote. To read the header of a single file:

hf download bartowski/gemma-2-9b-it-GGUF gemma-2-9b-it-Q4_K_M.gguf
python gguf-py/gguf/scripts/gguf_dump.py --no-tensors <path printed above>

The name is a summary written by a person. The header is written by the tool. When they disagree, the header is right — and if you are about to trust the file at all, it is worth checking its checksum against the published one before you load it.