Loading Two Models at Once in LM Studio

LM Studio will happily hold several models in memory at the same time. Whether that works on your machine is not a question about LM Studio; it is a sum of two terms per model, and both are calculable before you click anything.

How more than one model stays loaded

Each loaded model is an instance: its own copy of the weights, its own KV cache, its own context length, addressed by an identifier. Loading a second model does not replace the first unless something evicts it. From the CLI that identifier is explicit, which is also how you load the same model twice with different settings — one instance at a short context for classification, one long for summarising:

lms load qwen3-8b --context-length 4096  --gpu max --identifier fast
lms load qwen3-8b --context-length 32768 --gpu max --identifier long

lms ps        # what is loaded right now
lms unload fast

The --identifier flag is documented on LM Studio’s lms load reference as “the identifier to assign to the loaded model for API reference”, and it is the string you then put in the model field of a request to the local server on port 1234. Two instances of one file are two full copies in memory: the weights are not shared between them.

Everything currently resident is listed by the server’s /v1/models endpoint, which is the quickest way to check what a client can address without opening the app. The important detail is what happens when a request names something that is not in that list: rather than returning a 404, LM Studio may load it, which is the just-in-time behaviour described below. A misspelled model name in a config file can therefore cost you a multi-gigabyte load rather than an error, and on a machine that was already near its limit that is the load that fails.

The weights, derived

Take a model whose numbers are public. Alibaba’s Qwen3-8B lists 8,190,735,360 parameters in its Hugging Face repository metadata. A GGUF of it at Q4_K_M — the file published in bartowski’s Qwen_Qwen3-8B-GGUF repository — is 5,027,784,224 bytes. Divide:

5,027,784,224 bytes x 8 bits  = 40,222,273,792 bits
40,222,273,792 / 8,190,735,360 params = 4.91 bits per weight

so, for any model at this quantisation:
  weights on disk ~= params x 4.91 / 8 bytes

That figure is what Q4_K_M costs in practice rather than the nominal four bits, because the mixed k-quant scheme stores some tensors at higher precision and carries per-block scales. The assumption in the division is that the file is entirely weights and metadata, which for a GGUF is very nearly true. Loading is a memory-map of that file, so resident memory for the weights is approximately the file size, not less.

The KV cache, derived

The second term is the one people leave out, and it is the one that scales with your context slider. Qwen3-8B’s published config.json gives 36 hidden layers, 8 key-value heads and a head dimension of 128, with grouped-query attention meaning the cache is sized by the 8 key-value heads rather than the 32 attention heads. The cache stores a key and a value vector per token per layer:

bytes per token
  = 2 (K and V) x layers x kv_heads x head_dim x bytes_per_element
  = 2 x 36 x 8 x 128 x 2        (fp16 cache)
  = 147,456 bytes = 144 KiB per token

at  8,192 tokens : 147,456 x 8,192  = 1,207,959,552 B = 1.125 GiB
at 40,960 tokens : 147,456 x 40,960 = 6,039,797,760 B = 5.625 GiB

Stated assumptions: an fp16 cache at two bytes per element, no cache quantisation, and one sequence. Halve it if you run an eight-bit cache; multiply it by the number of concurrent sequences if you serve more than one at a time. The upper figure uses 40,960 because that is the max_position_embeddings in the same config — it is the model’s ceiling, not a number anyone invented.

Put the two together and a single Qwen3-8B instance at Q4_K_M with a 32k window needs roughly 5.0 GiB of weights plus about 4.4 GiB of cache. Two such instances need both twice. That is why a 12 GB card that comfortably runs one 8B model at a long context refuses the second, and why dropping the second instance to a 4k window — about 0.55 GiB of cache — often makes it fit. More on the cache itself in the KV cache explainer.

JIT loading, auto-evict and TTL

LM Studio will also load and unload models for you, which changes what “two models at once” means. Its documentation on idle TTL and auto-evict describes three behaviours worth knowing before you plan a budget:

• JIT loading loads a model on the first request that names it, so a client can switch models without you touching the app.
• Auto-evict, enabled by default and toggled under Developer → Server Settings, unloads the previously JIT-loaded model before loading a new one. With it on you effectively have one JIT model at a time regardless of how much memory is free. Turn it off and two JIT models coexist — and your budget is now the sum above.
• Idle TTL defaults to 60 minutes for JIT-loaded models: no requests for that long and the model is unloaded. Models loaded with lms load have no TTL by default and stay until you unload them. Set one per model with --ttl in seconds, or per request with a ttl field in the body.

Making two models coexist

1. Compute the budget before loading. Weights from the GGUF file size, cache from the formula above at the window you intend to set. Sum over every instance you want resident.
2. Give the small model the small window. A 1–4B model used for routing, classification or embeddings does not need 32k, and the cache saving is linear.
3. Load explicitly with --identifier rather than relying on JIT, so nothing is evicted behind your back, and disable auto-evict if you genuinely want both resident.
4. Give long-lived instances a TTL anyway. A model that stays resident forever on a shared workstation is memory nobody can reclaim, and --ttl 3600 costs one reload an hour at worst.
5. Watch the offload. If the second load pushes layers off the GPU it will still work and will be dramatically slower — check with lms ps and with GPU monitoring rather than by feel.