Ollama, llama.cpp and LM Studio Compared

The usual framing — which of these three is fastest — is answering a question that mostly does not exist, because two of them are wrappers over the same inference engine. The differences that matter are about defaults, packaging and what happens when you need a knob.

Three things, not three competitors

• llama.cpp is the engine: a C++ implementation of transformer inference with CPU and GPU backends, the GGUF format, and a bundled OpenAI-compatible server (llama-server) and benchmark tool (llama-bench). Everything is a flag. Nothing is chosen for you.
• Ollama is a model manager and daemon. It handles downloading, storage, templates and a small HTTP API, and runs the model through an engine descended from llama.cpp. Its value is that one command works and keeps working.
• LM Studio is a desktop application: a model browser, a chat interface, and a local server you can point other software at. Same engine lineage, a graphical front end, and a settings panel instead of flags.

So the throughput of all three on the same model, same quant, same context and same offload settings should land in the same neighbourhood, and where it does not, the cause is almost always a default that differs — context length, KV cache type, how many layers were offloaded, or whether flash attention was enabled. That is worth knowing before you spend an evening benchmarking wrappers.

What the engine decides

There is a genuinely different tier of tool that these three are not in, and choosing wrongly between the tiers costs far more than choosing wrongly within one. Engines like vLLM, SGLang and TGI are built for concurrent serving: continuous batching, paged KV cache, tensor parallelism across cards. They generally want unquantised or GPU-native quantised weights rather than GGUF, and they want a real GPU.

The split is about who the model is for. One person at a keyboard is a batch size of one, and llama.cpp’s lineage is optimised for exactly that, including on hardware without much of a GPU. Ten people at once is a scheduling problem, and a batching server will serve them on hardware that llama.cpp would serialise. Pick the tier by concurrency first, then pick within it.

A decision procedure

Answer in order; stop at the first yes.

• Do several people or processes hit this at once? Then you want a batching server, not any of these three. See the capacity model in the team-serving page.
• Do you want a graphical way to try models and compare prompts? LM Studio. The browse-download-chat loop is its entire reason to exist and it is genuinely the fastest way to answer “is this model any good at my thing”.
• Do you want other software on the machine to call a model, managed like a service? Ollama. A daemon, an API, a model store, and one command to update.
• Do you need a specific flag — an unusual cache type, a particular rope scaling, a draft model for speculative decoding, a build against a specific backend? llama.cpp directly. This also becomes the answer the moment you are debugging, because a wrapper hides the setting you need to see.

Nothing stops you from having two. A common and sensible arrangement is LM Studio for exploration and a llama.cpp or Ollama server for the thing you actually built.

A comparison harness you can run

If you do want to compare them on your own machine, the whole game is controlling the variables. Pin these five and the result means something; leave any of them floating and it does not.

• The identical GGUF file, by SHA, not “the same model”. Different repackagings of one model differ in quant mix.
• The same context length, set explicitly in all three. Defaults differ and the KV cache is a large share of memory.
• The same KV cache precision.
• Full GPU offload in all three, or the same partial split. Partial offload dominates every other effect.
• A warm run. Discard the first invocation entirely — it includes loading the file from disk.

# 1. baseline, engine directly
llama-bench -m ./m-Q4_K_M.gguf -p 512 -n 128 -ngl 99

# 2. the same file through Ollama, imported rather than re-downloaded
printf 'FROM ./m-Q4_K_M.gguf\nPARAMETER num_ctx 8192\n' > Modelfile
ollama create bench -f Modelfile
ollama run bench --verbose "write 128 words about tides"
#    --verbose prints eval counts and durations; divide to get tok/s

# 3. LM Studio: load the same file, set context to 8192,
#    enable its local server, then hit it the same way you would
#    any OpenAI-compatible endpoint and time the stream yourself.

Report prefill and generation separately, and report the second run. If the three numbers come out within a few per cent of each other, that is the expected result and you have learned something useful: choose on ergonomics, because performance is not the axis.

The differences that actually bite

• Default context length. Wrappers often default to a small window regardless of what the model supports, and long inputs are silently truncated. Set it explicitly and verify.
• Chat templates. The template turns your messages into the exact token sequence the model was tuned on. Wrappers ship their own copies, and a subtly wrong template degrades quality in a way that looks like a bad model. GGUF files can carry the correct template as metadata — prefer that over an override.
• Model storage and disk. Each tool keeps its own directory. Three tools and a habit of trying things is tens of gigabytes before you notice. Import one file into all three rather than downloading three copies.
• API surface. All three expose something OpenAI-shaped, but coverage of tool calling, structured output and log probabilities varies by version. Test the specific field your application depends on rather than assuming compatibility.
• Where the flags went. When something is slow or wrong, the debugging path runs downward through the layers. Being able to drop to llama-server with explicit flags and reproduce the problem is the fastest way to find out whether the engine or the wrapper is at fault.