llama.cpp's mmap and mlock Flags Explained

Memory-mapping is why a 4 GB GGUF can appear to load in under a second and why two llama.cpp processes on the same model do not use twice the RAM. It is also why your resident-memory number is not the number you think it is.

What mmap actually does with a GGUF

Reading a file normally means the kernel pulls it into the page cache and then copies it into your process’s heap: two copies, and you wait for all of it before anything runs. mmap instead maps the file’s pages directly into the process’s address space. No copy is made, nothing is read until a page is touched, and the pages that are read are the same page-cache pages the kernel would have used anyway.

GGUF is designed for this. Tensors are stored contiguously and aligned, so a tensor in the file is a pointer into the mapping rather than something that has to be unpacked. Three consequences follow:

• Load looks instant and is not. The mapping is established immediately; the reading happens as faults during the first pass over the weights. The cost moved, it did not vanish, and it shows up as a slow first token rather than a slow start.
• Two processes share one copy. Two servers on the same GGUF map the same page-cache pages. This is the single strongest argument for mmap on a machine running more than one model process — a benchmark run alongside a live llama-server costs one copy, not two — and it is why a naive sum of resident sizes overcounts.
• Pages are evictable. Because they are clean file pages, the kernel can drop them under pressure and re-read them from disk later. No swap is involved — the file is the backing store — which is both the feature and the problem.

Both flags are now spellings of --load-mode

This is the part most existing writing on the subject has not caught up with. On current llama.cpp master, --mlock and --no-mmap are marked DEPRECATED in favour of -lm/--load-mode, and each prints a warning telling you the replacement. The old flags still work; they simply set the new parameter.

-lm, --load-mode MODE      model loading mode (default: auto)
  auto        mmap, unless a device does not support it
  none        no special loading mode
  mmap        memory-map model
  mlock       force system to keep model in RAM rather than swapping
  mmap+mlock  mmap plus the mlock guarantee
  dio         use DirectIO if available

Note mmap+mlock: the two were never mutually exclusive, and the combined mode is what people setting both old flags together were usually after. There is also a warning if you mix --load-mode with any of the deprecated flags on one command line, because only the last one on the line takes effect — an easy way to silently get a mode you did not choose when a script appends flags.

What mlock buys, and what it risks

mlock asks the kernel to pin pages in physical memory so they cannot be evicted. Applied to a mapped model, it converts “these pages are probably resident” into “these pages are resident”. What you buy is the elimination of a specific, ugly failure: a model that was fast for an hour becomes slow because something else on the machine caused the kernel to evict half its weights, and every subsequent token pays disk reads it did not pay before.

What you risk is everything else on the machine. Pinned pages are not available to anyone, so if the model plus your working set exceeds physical RAM, the pressure lands on processes that can still be evicted or swapped. On a desktop this is a stuttering desktop; on a server it can be the OOM killer choosing something you cared about. There are also limits: on Linux, pinning beyond RLIMIT_MEMLOCK fails, and the failure is a warning at load rather than a refusal to run, so check the log rather than assuming it took.

The honest rule is that mlock is for a machine dedicated to the model. If the box does one job and has comfortably more RAM than the model needs, pinning removes a variance you would otherwise chase. If it is a laptop that also runs a browser, it is a way to make the browser somebody else’s problem.

When mmap is the wrong choice

The default is auto — mmap unless a device does not support it — and the exceptions are worth naming because each has a distinct symptom.

• Full GPU offload. When every layer goes to device memory, the weights are copied to VRAM during load and the host mapping is not on the hot path afterwards. Mapping still avoids a host-side copy on the way, so it rarely hurts, but the sharing argument disappears — two processes do not share VRAM. See the -ngl page for what “every layer” actually means.
• Network or slow filesystems. A GGUF on NFS or a spinning disk turns every page fault into a slow read at unpredictable moments. Loading it eagerly instead gives you one long wait rather than a permanently jittery run.
• Tight memory with a large model. Under real pressure, mapped pages thrash: read, evict, read again. Throughput collapses without any error appearing. This is the case mmap+mlock exists for — and if the model genuinely does not fit, the answer is a smaller quantization, not a flag.
• Windows. The mapping semantics differ enough that behaviour under memory pressure is not the same as on Linux; treat timings measured on one as not transferable to the other.

dio is the newest option: DirectIO bypasses the page cache entirely, reading into the process without the kernel keeping a copy. On a machine where the page cache is being churned by other work that is a real win, and on a machine that reloads the same model repeatedly it is a loss, because there is no cached copy for the next start.

Choosing, by what you are optimising

• Fast repeated starts, several models, shared box — auto. The page cache does the work and nothing is pinned.
• Stable tokens per second on a dedicated machine — mmap+mlock, with enough RAM headroom that pinning is not a gamble. Confirm the log shows no warning about the lock limit.
• Model on a network share — eager loading, and better yet, copy it locally first.
• Heavy competing I/O — try dio and compare steady-state throughput, not load time.

Whichever you choose, measure the thing you care about. Load time and steady-state throughput move in opposite directions here, and a flag that halves one while degrading the other is a bad trade if you start the process once a week. For steady-state comparisons, llama-bench takes -mmp 0,1 and will run both.