Budgeting Disk Space for Keeping Several Local Models Around

Model files are advertised in round numbers and stored in exact ones, and the exact ones are what fills a drive. Here is a real library totalled from published file sizes, and the three places extra copies appear.

What one model costs

Every figure below is the byte size published in the corresponding Hugging Face repository’s file listing, read on 2026-08-11. They are all Q4_K_M, the most commonly downloaded level, from bartowski’s GGUF repositories.

Phi-3.5-mini-instruct-Q4_K_M.gguf          2,393,232,672 B   2.23 GiB
Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf    4,920,739,232 B   4.58 GiB
gemma-2-9b-it-Q4_K_M.gguf                 5,761,057,728 B   5.37 GiB
Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf    8,988,111,072 B   8.37 GiB
Meta-Llama-3.1-70B-Instruct-Q4_K_M.gguf  42,520,398,400 B  39.60 GiB

Note the two units, because this is where budgets go wrong. Drive capacity is sold in decimal gigabytes and reported by most operating systems in binary gibibytes. The 70B file is 42.5 GB by the marketing measure and 39.6 GiB by the one your file manager shows. The file did not change; the divisor did.

A five-model library, totalled

Adding those five gives 64,583,539,104 bytes: 60.15 GiB, or 64.58 GB. That is a plausible library — a tiny model for quick classification, a general 8B, a 9B alternative with a different character, a coding model, and one large model for the jobs the others fail.

The distribution matters more than the total. The 70B alone is 39.60 GiB of that 60.15 GiB — 66% of the library for one file. Drop it and the other four come to 20.55 GiB, which fits comfortably on almost any machine. That single decision, rather than any per-file optimisation, is the disk budget.

A second thing the total hides: large models arrive as shards. The same repository ships Llama-3.1-70B at Q5_K_M as two files of 37.14 GiB and 9.38 GiB. You need both, and a download interrupted after the first one leaves you 37 GiB of disk consumed by something that will not load.

A third: the weights file is not always the whole model. A vision model needs its mmproj projector alongside the language weights, which is a separate file in the same repository. A model you intend to serve with an adapter needs the adapter too. And if you plan to keep an importance matrix for future quantization work, that is another file per model. None of these are large next to the weights, but they are the reason a directory you sized at exactly 60 GiB comes up short.

The largest single omission from most budgets is working space. If you ever intend to quantize a model yourself rather than downloading a finished quant, you need the original checkpoint and the converted GGUF and the output resident at the same time. A BF16 checkpoint is two bytes per parameter, so an 8B model is roughly 16 GB of source, another 16 GB of converted GGUF, and 4.9 GB of output — about 37 GB of peak usage to produce a 4.9 GB file. Budget for the peak, not the result, because running out halfway through a conversion leaves you with all three partial artefacts and no model.

The copies you did not ask for

The sum of the files is a floor, not the total. Three mechanisms routinely put the same weights on disk twice:

• The Hugging Face cache on Windows. The cache normally stores each file once in blobs/, named by its hash, and points at it from snapshots/<commit>/ with a symlink. Hugging Face’s cache documentation records that where symlinks are unavailable — the known case being Windows without Developer Mode or administrator rights — the library skips blobs/ and writes files directly into snapshots/. Two revisions of a repository then mean two full copies of every file they share. On a 40 GB model that is 40 GB.
• Tool-specific stores. Ollama keeps its own blob store, at ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux and C:\Users\%username%\.ollama\models on Windows, relocatable with OLLAMA_MODELS. It deduplicates within itself but knows nothing about a GGUF you also downloaded for llama.cpp. Pull the same model into both and you own it twice.
• Self-contained executables. A llamafile embeds the weights in the binary, so a portable llamafile setup is another full copy unless you deliberately use the external-weights form.

There is also a fourth, smaller one. Hugging Face’s Xet transfer layer keeps a chunk cache under ~/.cache/huggingface/xet, documented with a 10 GB limit for the download chunk cache and a 4 GB soft limit for the upload shard cache. It is bounded, but it is not nothing, and it is invisible to anything that only counts model files.

What a higher quant level costs you

The same four small models at Q8_0, using published sizes from the same repositories:

                      Q4_K_M      Q8_0
Phi-3.5-mini          2.23 GiB    3.78 GiB
Llama-3.1-8B          4.58 GiB    7.95 GiB
gemma-2-9b-it         5.37 GiB    9.15 GiB
Qwen2.5-Coder-14B     8.37 GiB   14.62 GiB
                     ---------   --------
                     20.55 GiB   35.51 GiB   = 1.73x

The multiplier is close to the ratio of bits per weight — Q8_0 is 8.5 bits against Q4_K_M’s roughly 4.89, or 1.74 — which is the check that the file sizes are behaving as the encodings say they should. It costs about 15 GiB to hold this library at Q8_0 instead of Q4_K_M, and whether that is worth it is a per-task question covered in choosing a quantization level by use case.

The trap to avoid is keeping several quants of the same model “to compare”. Three levels of one 14B model is 20 to 30 GiB spent on a comparison you will make once. Download two, decide, delete one.

Finding and reclaiming the space

Both major tools can report and prune their own stores. The Hugging Face CLI aggregates by repository and accepts filters on size and last access:

hf cache ls
hf cache ls --revisions --filter "size>1GB" --filter "accessed>30d"

# delete everything untouched for a year, no prompt
hf cache rm $(hf cache ls --filter "accessed>1y" -q) -y

# remove unreferenced revisions and leftover .incomplete downloads
hf cache prune

# ollama's own store
ollama list
ollama rm qwen2.5-coder:14b

hf cache prune is the one worth running first: it removes revisions no branch or tag points at any more, plus the .incomplete partial blobs left by interrupted downloads, which are exactly the 37 GiB half-shard described above and are not counted by an ordinary cache listing.

One caution before you delete anything from blobs/ by hand: the snapshot entries are symlinks into it, so removing a blob leaves a dangling link that fails at load time with a confusing error rather than a missing-file one. Use the tools, which understand the reference counting.