Running Ollama on Windows via WSL2

Ollama has had a native Windows build for a long time, so installing it inside WSL2 is a choice rather than a workaround. The choice is worth making for some workloads and not others, and either way the part that goes wrong is the same: the install succeeds, the model answers, and it is answering on the CPU.

When WSL2 is the right choice

The honest reasons to put Ollama inside WSL2 are all about what is around it. If the code calling Ollama is a Linux container, a Python environment built against Linux wheels, or a service you intend to run on a Linux box later, running the model server on the same side of the boundary removes a class of path and networking problems you would otherwise debug twice. If you want the systemd unit — and therefore systemctl edit ollama.service as the place environment variables live — that is a Linux install.

The reason not to is duplication. A native Windows Ollama and a WSL2 Ollama are two servers, two model stores, and two claims on port 11434. The Windows one keeps its blobs under C:\Users\%username%\.ollama\models and the Linux service under /usr/share/ollama/.ollama/models, per Ollama’s FAQ, so a model you pulled on one side is invisible to the other and you have paid for the download twice. Pick one and uninstall the other, or at minimum give them different ports.

The driver goes on Windows, not in WSL

This is the instruction people skip because it is counter-intuitive, and it is the one that produces a broken install. NVIDIA’s CUDA on WSL user guide is explicit that the Windows driver is the only driver you need to install and that you must not install any Linux display driver inside WSL 2 — the Windows driver is projected into the distribution as a stub libcuda.so. Installing a Linux nvidia-driver-* package on top of that is what breaks it, and the failure looks like a missing GPU rather than a conflict.

Two consequences follow. NVIDIA’s guide notes that a root user on bare metal will not find nvidia-smi at the usual path and should use /usr/lib/wsl/lib/nvidia-smi or add /usr/lib/wsl/lib/ to PATH. And your card has to be new enough: Ollama’s GPU documentation states support for NVIDIA GPUs with compute capability 5.0 and above and driver version 550 or newer, with cards in the 5.0–6.2 range needing 570 or newer. A Kepler card will never be accelerated here, and no amount of reinstalling changes that.

Installing Ollama in the distribution

1. Confirm you are on WSL 2, not WSL 1. wsl -l -v in PowerShell prints a VERSION column; GPU compute requires 2. Convert with wsl --set-version Ubuntu 2 if needed.
2. Install or update the NVIDIA driver on the Windows host from NVIDIA’s own downloads, then reboot Windows. Nothing in the next steps can succeed before this one has.
3. Inside the distribution, check the projection worked before installing anything else: /usr/lib/wsl/lib/nvidia-smi. You want a table with your card and a driver version in it. If this fails, stop — installing Ollama will not fix it.
4. Install Ollama with the official script: curl -fsSL https://ollama.com/install.sh | sh. On a distribution with systemd enabled it registers and starts an ollama.service unit running as the ollama user.
5. If your distribution does not have systemd enabled, add it to /etc/wsl.conf and restart the distribution from Windows with wsl --shutdown:

   [boot]
   systemd=true

   Without systemd there is no unit, so you run ollama serve in a terminal yourself and it dies when the terminal does.
6. Pull something small enough to load entirely on the card, so that the next section’s signals are unambiguous: ollama pull llama3.2:3b.

Proving the GPU arrived

Ollama falls back to CPU silently and cheerfully. A 3B model on a modern CPU still answers in a couple of seconds, which is exactly fast enough for you to believe it worked. Check three independent signals rather than one.

• The scheduler’s own answer. Start a generation, and while it is loaded run ollama ps in another shell. The PROCESSOR column reads 100% GPU when every layer is resident on the card, and something like 48%/52% CPU/GPU when it is split. The same information is in the JSON from /api/ps as size against size_vram — if size_vram is zero or much smaller than size, the card is barely involved.
• The server log at load time. journalctl -u ollama -f while you load a model. A healthy boot names the library it selected and the VRAM it found. The failure string to search for is no compatible GPUs were discovered, which is what Ollama logs before deciding to run on the CPU.
• The driver’s view. /usr/lib/wsl/lib/nvidia-smi during generation should show anollama process and non-trivial memory in use. This one is worth having because it is outside Ollama entirely — ifnvidia-smi sees the work and ollama ps does not, the problem is your reading of the output, not the passthrough.

If the model is splitting across CPU and GPU rather than failing outright, the cause is usually memory rather than plumbing: weights plus KV cache did not fit, so the scheduler kept some layers in system RAM. That is the same arithmetic covered in how Ollama picks how many layers to offload, and shrinking the requested context is often enough to tip it back.

Reaching it from Windows

By default the server binds to 127.0.0.1:11434, and inside WSL2 that means the loopback interface of the distribution. WSL’s localhost forwarding usually makes http://localhost:11434 work from a Windows browser anyway, but it is not something to build on: it depends on your networking mode, and it does not help another machine on the LAN at all.

The reliable version is to bind explicitly. Set OLLAMA_HOST=0.0.0.0:11434 in the service environment — under systemd that is a systemctl edit ollama.service drop-in with an Environment= line, followed by systemctl daemon-reload and a restart. If a browser page is going to call it directly you also need OLLAMA_ORIGINS set to the origin that page is served from, because the default allow-list is local only. Both variables, and the places each platform expects them, are covered in Ollama’s environment variables.

One last collision to know about: if the native Windows Ollama is also installed and running, it already holds 11434 on the Windows side, and requests you think are going to WSL2 are being answered by a different server with a different model store. ollama list returning models you do not remember pulling is the tell.