Running Ollama on a Raspberry Pi

Installing Ollama on a Pi is one command. Everything that decides whether it is usable happens afterwards, in the daemon configuration and in where the model files live.

Before you install

Ollama’s Linux builds are x86-64 and ARM64. A Pi running a 32-bit OS has no binary to install, and this is the most common first failure. Check:

uname -m        # must print aarch64
free -h         # how much RAM you actually have
df -h /         # how much space on the root filesystem

Then size the model to the machine. Producing one token reads every weight, so a model whose weights do not fit in RAM is not slow, it is thrashing. On a 4 GB Pi that means a 1B or 3B model at 4-bit; on 8 GB a 7B or 8B fits its weights with a little room; on 16 GB you have genuine headroom for context. The arithmetic behind those brackets, including what the KV cache costs per token, is worked through in the memory budget for an SBC and applies unchanged here.

Installing

1. Run the install script. It detects ARM64, places the binary in /usr/local/bin/ollama, creates an ollama system user and installs a systemd unit.

   curl -fsSL https://ollama.com/install.sh | sh

   Read the script before piping it to a shell if that matters to you; it is plain text at that URL.
2. Check the service.

   systemctl status ollama
   curl http://127.0.0.1:11434/api/tags

   An empty model list from that endpoint means the daemon is up.
3. Pull a small model and talk to it.

   ollama pull llama3.2:1b
   ollama run llama3.2:1b "Summarise this in one sentence: ..."

Do not pull the model before doing the next section. A multi-gigabyte download onto the SD card is exactly the write pattern you are trying to avoid, and moving the store afterwards means writing it twice.

Move the model store first

Ollama’s documentation gives the default model directory on Linux as /usr/share/ollama/.ollama/models, which on a Pi is the SD card. Models are stored as content-addressed blobs, so every pull writes the whole model once and every re-pull of a changed tag writes it again. Put that on a USB SSD, both for the write endurance and because the model is read from disk on every load.

sudo mkdir -p /mnt/ssd/ollama-models
sudo chown -R ollama:ollama /mnt/ssd/ollama-models
sudo systemctl edit ollama.service

In the override editor that opens, add:

[Service]
Environment="OLLAMA_MODELS=/mnt/ssd/ollama-models"

sudo systemctl daemon-reload
sudo systemctl restart ollama

The chown matters: Ollama’s documentation notes the ollama user needs read and write permission on a custom directory, and a permission failure here presents as a pull that starts and then dies. Make sure the SSD is mounted at boot via /etc/fstab before the service starts, or the daemon will helpfully create an empty directory on the mount point instead.

Swap, and why it is a trap

Raspberry Pi OS ships with a small swap file managed by dphys-swapfile, and guides routinely tell you to enlarge it so a model that does not fit will still load. It will load. It will also be unusably slow, because a swapped-out weight page has to come back from storage on every single token, and storage is three orders of magnitude slower than DRAM.

Treat swap as a safety margin for the loading spike and for everything that is not the model, not as extra RAM:

sudo dphys-swapfile swapoff
sudo sed -i 's/^CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile
sudo dphys-swapfile setup
sudo dphys-swapfile swapon
free -h

Two follow-ups. Point CONF_SWAPFILE at the SSD as well if you have one — sustained swap on an SD card is the single fastest way to wear one out, for reasons covered in what storing model files does to an SBC’s SD card. And consider zram instead, which compresses pages in RAM and writes nothing: it buys less headroom than a swap file but costs no endurance and no I/O.

If a model needs swap to load at all, the answer is a smaller model or a smaller quantization, not a bigger swap file.

Daemon settings that matter

These are all documented environment variables, set the same way as OLLAMA_MODELS above — via systemctl edit ollama.service, one Environment= line each, then daemon-reload and restart. From Ollama’s documented FAQ:

• OLLAMA_CONTEXT_LENGTH — documented default 4096 tokens. This is the single largest memory lever after the model itself, because the KV cache grows linearly with it. Lower it to 2048 on a 4 GB board; raise it only when you have measured the headroom.
• OLLAMA_KEEP_ALIVE — how long a model stays resident, default 5 minutes. On a Pi, loading a model from storage is slow enough that a longer value is usually right for an always-on service, and 0 is right if you need the RAM back between requests.
• OLLAMA_NUM_PARALLEL — parallel requests per model, documented default 1. Leave it there. Concurrency on a bandwidth-bound machine does not increase total throughput; it multiplies the KV cache and makes every request slower.
• OLLAMA_MAX_LOADED_MODELS — documented default 3 on CPU. On a Pi that is three ways to run out of memory; set it to 1.
• OLLAMA_KV_CACHE_TYPE — documented default f16. Quantizing the KV cache halves its memory cost at some quality risk, and on a memory-limited board it is the lever that buys context length.
• OLLAMA_HOST — documented default 127.0.0.1:11434. Setting it to 0.0.0.0:11434 exposes the API to your network with no authentication of any kind, so put it behind something before you do that.

Two habits make the rest of this bearable. ollama ps shows which models are resident and how much they are holding, which is the fastest way to find out that a model you finished with an hour ago is still occupying the RAM you now need. And the first request after a model has been evicted pays the full load from storage — several seconds on an SSD and considerably more on an SD card — so a service that feels instant in testing and slow in production is usually one whose OLLAMA_KEEP_ALIVE expires between real users. Send a one-token request on a timer if you need the model warm, rather than raising the keep-alive to infinity on a machine that has other work to do.

For the same hardware driven directly, with a benchmark harness that separates prompt processing from generation, see running a small chat model on a Raspberry Pi with llama.cpp.