Ollama's keep_alive Setting and Model Unloading

A model that has just answered you is still in memory. Five minutes later it is not, and the next request pays the load again. That five minutes is keep_alive, and understanding it as an idle timer — reset by every request, not counted from load — explains most of what looks like inconsistent latency on a local setup.

It is an idle timer, not a lifetime

Ollama’s scheduler tracks a reference count per loaded runner. While a request is in flight the count is above zero and no timer exists. When the last request finishes and the count reaches zero, the scheduler starts a timer for the session duration; if another request arrives first, the timer is reset rather than allowed to run. The runner is only unloaded when the timer fires with the model still idle.

So a model serving one request a minute with a five-minute keep-alive never unloads, and a model serving one request an hour reloads every time. The cost of that reload is reading the weights from disk into VRAM — for an 8B model at a 4-bit K-quant that is roughly five gigabytes of I/O, which is seconds on NVMe and considerably longer on a spinning disk or a network share. Ollama reports it separately: the load_duration field in a generate or chat response is nanoseconds spent loading, distinct from prompt_eval_duration and eval_duration. A request whose load_duration is large is a cold start, and no amount of prompt tuning will fix it.

Every value it accepts

The keep_alive field is accepted by /api/generate, /api/chat and the embedding endpoints, and Ollama’s FAQ documents four forms:

• A duration string — "10m", "24h". Go duration syntax, so "90s" and "1h30m" both parse.
• A number of seconds — 3600, unquoted. This is the form that trips people up in shells and templating languages, because "3600" as a bare quoted string is not a duration.
• Any negative number — -1 or "-1m". Keeps the model loaded indefinitely. There is no separate “pin” API; this is it.
• Zero — unload as soon as this request finishes. The scheduler treats a non-positive duration as “expire on idle” and skips the timer entirely.

The two extreme values have a use beyond configuration. Sending a request with only a model field and "keep_alive": -1 preloads and pins a model without generating anything; the same request with 0 unloads it. On the CLI, ollama stop llama3.2 does the second one, and ollama run llama3.2 "" does a plain preload.

The trade you are making

The quantity you are renting is the resident size of the model, which ollama ps prints in its SIZE column and /api/ps returns as size and size_vram per model. That is weights plus the KV cache for the context it was given, so it is larger than the file on disk, and it grows with num_ctx.

The quantity you are buying is load_duration on the first request after each idle gap. Which way that trade goes is decided by your own traffic and nobody else’s, and it is worth doing the arithmetic explicitly rather than picking a number that feels safe. Divide your day into idle gaps longer than the keep-alive: that count is how many cold starts you will pay. If the gaps are mostly shorter than five minutes, a longer keep-alive buys nothing and costs the whole resident size. If your traffic is a handful of requests spread across the day, a long keep-alive holds gigabytes hostage to avoid a delay nobody is waiting on. The awkward middle — bursts a few times an hour — is where a keep-alive slightly longer than your burst gap does real work.

Concurrency changes the sums. Multiple models can be resident at once, bounded by OLLAMA_MAX_LOADED_MODELS, and each holds its own keep-alive timer. Pinning two models with -1 on a machine that can hold one means the scheduler evicts to make room and your pin becomes a suggestion, which is covered in running multiple models simultaneously.

Which setting wins

There are three sources and the order is documented. The keep_alive field on a request overrides everything for that load. Failing that, the OLLAMA_KEEP_ALIVE environment variable on the server sets the default, and it takes the same four value forms. Failing that, the built-in default is five minutes, defined in Ollama’s configuration source as a five-minute duration.

The variable is read by the server process, not the client, which is the single most common reason a change appears to do nothing. On Linux under systemd that means an Environment= line in a systemctl edit ollama.service drop-in followed by a daemon reload; on macOS, launchctl setenv and an application restart; on Windows, the user or system environment and a restart from the Start menu. Setting it in the terminal where you run curl changes nothing at all. The full list is in Ollama’s environment variables.

Patterns that hold up

• An interactive assistant on a workstation. Leave the default. Bursty human use resets the timer constantly, and the memory returns to you when you stop working.
• A background job on a shared box. Send "keep_alive": 0 on the last request of the batch so the model is gone the moment the work is, rather than sitting on VRAM somebody else is queueing for.
• A latency-sensitive endpoint. Pin with -1 and preload on service start with an empty generate request, so the first real user is not the one who pays the load.
• A machine that runs several models. Give the small frequently used one a long keep-alive and the large occasional one a short one, rather than raising the global default and letting the scheduler thrash.