The Difference Between Loading and Warming Up a Local Model

“The model is loaded” describes at least two different states, and a request arriving in the first one behaves very differently from a request arriving in the second. The distinction is not pedantry — it is the reason a server that reports itself ready still answers the first question slowly, and the reason a flag that makes startup faster makes your users wait longer.

Two phases that get called one thing

Loading is getting weights from a file into a place the compute device can read them: opening the GGUF, parsing its metadata and tensor table, deciding how many layers go on the GPU, and arranging for the bytes to be reachable. When this finishes, the runtime knows the model’s shape and has committed the memory for it.

Warming up is running one forward pass through the whole thing before anybody asks for anything. That pass is what allocates the KV cache and the compute buffers, forces every weight to be genuinely present rather than merely addressable, causes the compute library to select and load kernels for the shapes it is about to see, and initialises the handles those libraries keep.

Both phases are one-time, but they are one-time in different scopes. Loading is per model instance. Some of what warmup triggers — driver context creation, kernel module loading — is per process, which is why the second model loaded into a running server starts faster than the first.

Memory mapping is why “loaded” is ambiguous

llama.cpp maps the GGUF file into the address space rather than reading it. The kernel sets up a mapping and returns immediately; nothing has been read from the disk yet. The bytes arrive later, in page-sized pieces, as a page fault when something touches them.

This is a good design and it makes “loaded” ambiguous. Load reports as complete in a fraction of a second, and then the first forward pass takes several seconds while it faults in four gigabytes of weights a page at a time. The work did not disappear; it moved from a phase you were watching to a phase you were not.

It also produces the effect people notice most: the second load of the same file is dramatically faster, because those pages are still in the operating system’s page cache and the mapping resolves without touching the disk. That has nothing to do with the GPU, nothing to do with the model, and everything to do with the fact that your file is now in RAM. It also stops happening the moment the model is bigger than your free memory, at which point every load is a cold load and pages are being evicted while they are still needed. The arithmetic on how long each of those phases should take is a separate page.

What the warmup pass actually does

• Touches every weight. A forward pass reads every tensor, so every mapped page faults in exactly once. After this, the mapping is fully resident and no request pays for it again.
• Allocates the KV cache. The cache is sized from n_ctx and the model geometry, and it is committed on first use rather than at load. This is where a configuration that was going to run out of VRAM finds out.
• Allocates compute buffers. Scratch space for activations, sized for the largest batch the runtime expects. On a server with several slots this is meaningfully larger than on a single-user setup.
• Selects and loads kernels. Compute libraries pick an implementation based on the shapes they are given, and loading a compiled kernel module is not instant. The first call with a new shape pays it.
• Initialises library handles. Creating a cuBLAS handle, allocating its workspace, setting up streams — small individually, and all of it on the critical path of whichever request happens to be first.

Notice that the second and third items scale with your configuration rather than with your model. Raising -c or --parallelmakes warmup slower and does nothing to steady-state speed until a request actually uses the extra context.

The flags that move work between the phases

None of these make the total smaller. They decide who waits.

• --no-warmup — skip the pass entirely. Startup completes sooner and the first real request absorbs everything warmup would have done. Useful when you are cycling models during development and do not want to pay for a pass you may never use; wrong for anything serving traffic.
• --no-mmap — read the file into ordinary memory instead of mapping it. Load becomes slow and honest rather than fast and deferred, and the pages are yours rather than the page cache’s, so nothing can evict them. Loading a LoRA adapter disables mmap regardless, because the weights are being modified.
• --mlock — lock the mapped pages into physical memory so the kernel cannot page them out under pressure. This is the fix for a model that starts fast, runs fast, and then becomes slow an hour later on a machine that is doing other things. It requires the privilege to lock memory and it will fail loudly if it cannot.

What follows for how you run a server

The first consequence is about health checks. A process that has finished loading is not a process that can answer quickly, so a readiness probe that only checks the port is open will route traffic into the warmup window. Probe with something that actually generates a token, and let the warmup pass complete before you report ready.

The second is about unloading. Ollama unloads models after an idle period governed by OLLAMA_KEEP_ALIVE, which means a machine with sporadic traffic pays both phases repeatedly, and the “it is slow in the morning and fast in the afternoon” pattern is exactly that. If the traffic pattern justifies it, keeping the model resident costs some idle power and buys back both phases on every first request of a burst.

The third is about measurement. Any tokens-per-second figure taken from a first request after load is measuring warmup, not the model. Discard the first result, always — and if you are comparing two configurations, discard the first from each, or you are comparing page-cache states. A repeatable measurement starts after the model is warm and says so.

The last one is the least obvious. Because warmup allocates the KV cache and the compute buffers, a server that starts cleanly has already proved it can hold its configuration. A server started with --no-warmup has proved nothing, and an out-of-memory failure that would have happened at startup now happens on a user’s request instead. That alone is a reason to leave warmup on in anything that matters.