Accessing a Local Model Server From Another Device on Your Network

Making a local model reachable from your phone is one flag. The reason this page is longer than one flag is that the same server, by documented default, has no authentication of any kind — so the flag that makes it reachable from your sofa makes it reachable from every other device on the network at the same time, including ones you did not put there.

What the default actually is

Two defaults, both documented, and they only make sense together.

llama.cpp’s server README lists --host with a default of 127.0.0.1 and --api-key with a default of none. The llama.cpp server documentation is explicit on both. Bearer-token authentication is enforced only when you supply a key; without one, the server accepts every request it receives. The loopback bind is the only thing standing between that and the network, which is exactly why the loopback bind is the default.

Ollama is in a similar position with less to work with: it binds to loopback unless OLLAMA_HOST says otherwise, and it has no built-in API key mechanism at all. There is no flag that adds authentication. If you need it, it has to come from something in front of the server.

What an unauthenticated server exposes

It is worth knowing what the API surface actually permits before you decide the risk is acceptable, because it is more than free inference.

• Everyone else’s prompts. llama.cpp’s /slots endpoint is enabled by default and reports each slot and the content it is processing. Anyone who can reach the port can read what the server is being asked, which for a family or office machine is the most immediate problem on this list. --no-slots disables it.
• Server configuration. --props enables changing global properties over POST /props. It is disabled by default; leave it that way on anything bound to a network.
• Model management, on Ollama. The Ollama API includes endpoints to pull, create and delete models. An unauthenticated server on a reachable interface lets anyone who finds it make your machine download arbitrary weights — filling the disk — or delete the ones you have.
• Your compute, indefinitely. Even with everything above locked down, an open inference endpoint is free GPU time on your electricity bill for whoever finds it.

None of this is a defect in either project. Both are documented defaults that are correct for loopback. They stop being correct the moment the bind address changes, and nothing in the software will tell you that.

The procedure

1. Create a key first, before touching the bind address. Doing it in this order means the server is never reachable and open at the same time, even for the thirty seconds it takes you to type the next command.

   openssl rand -hex 32 > ~/.llama-api-key
   chmod 600 ~/.llama-api-key

2. Find the machine’s LAN address and subnet. You need both: the address to call, and the subnet to scope the firewall rule.

   ip -4 addr show scope global    # Linux  -> e.g. 192.168.1.42/24
   ipconfig getifaddr en0         # macOS
   ipconfig                       # Windows

3. Open exactly one port, to exactly that subnet. Not to everything. A rule scoped to the LAN subnet is what stops a VPN peer or a bridged guest network from reaching it.

   # Linux, ufw
   sudo ufw allow from 192.168.1.0/24 to any port 8080 proto tcp
   
   # Windows, PowerShell as administrator
   New-NetFirewallRule -DisplayName "llama-server LAN" \
     -Direction Inbound -Protocol TCP -LocalPort 8080 \
     -RemoteAddress 192.168.1.0/24 -Action Allow

4. Start the server bound to the network, with the key. Both flags in the same command, every time. Use --api-key-file rather than --api-key so the key does not appear in your shell history or in the process list, where any local user can read it from ps.

   llama-server \
     -m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
     -c 8192 -ngl all \
     --host 0.0.0.0 --port 8080 \
     --api-key-file ~/.llama-api-key \
     --no-slots

5. Confirm the key is enforced before you trust it. A request without the header must be rejected. If it is not, the key is not loaded and you have an open server.

   # From the server machine itself:
   curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models
   # expect 401
   
   curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models \
     -H "Authorization: Bearer $(cat ~/.llama-api-key)"
   # expect 200

6. Call it from the phone or the other laptop. This is the step the page promised:

   curl http://192.168.1.42:8080/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer PASTE_THE_KEY_HERE" \
     -d '{
       "model": "local",
       "messages": [{"role": "user", "content": "Name three tide-table terms."}],
       "max_tokens": 200
     }'

For Ollama the shape is the same minus step 1, because there is no key to create. OLLAMA_HOST=0.0.0.0:11434 ollama serve binds it; the firewall rule and the subnet scoping do all the work, and the authentication has to come from a reverse proxy in front of it. Do not skip that step on a network with devices you do not control.

Verifying it from the outside

Two checks, and the second is the one people skip.

The first is that an unauthenticated request from another device is refused. Run the curl from step 5 again from the phone, without the header, and confirm the 401 — a firewall rule and a key can both look correct and still not be applied to the interface you are testing.

The second is that the port is not reachable from the internet. If the router has UPnP enabled, or if somebody once set up port forwarding for something else on the same port, the server you just bound to 0.0.0.0 may be answering the world. Check from a connection that is not your own network — a phone on mobile data is the easiest — against your public address:

curl -s ifconfig.me            # your public IP, from the server machine

# then, from a device on mobile data (NOT your wifi):
curl -m 5 http://YOUR_PUBLIC_IP:8080/v1/models
# The only acceptable outcome is a timeout or a connection refusal.

Never deliberately forward this port. An inference server on the open internet will be found — scanners sweep the common ports for these services continuously — and a key over plain HTTP is transmitted in the clear anyway. If you need access from outside the house, the answer is in the next section, not in the router.

When not to bind at all

Binding to 0.0.0.0 is the right answer for a trusted home network with a scoped firewall rule and a key. There are two cases where it is the wrong answer and something else is easier.

If you only need it from one other machine, an SSH tunnel gets you there with no bind change, no firewall rule and encryption for free. The server stays on loopback and the tunnel forwards to it, so nothing on the LAN can reach it at all:

ssh -N -L 8080:127.0.0.1:8080 [email protected]
# the client now talks to localhost:8080 and the server never left loopback

If you need it from outside the network, use an overlay network rather than a port forward. A WireGuard or Tailscale interface gives the server an address that only enrolled devices can route to, and you bind to that interface’s address rather than to 0.0.0.0. That is a strictly better position than a forwarded port with a key, because the port is not reachable to be attacked in the first place.

One last note on encryption. Everything above is plain HTTP, which means your prompts and your API key are readable by anything that can observe the traffic — a concern on shared or wireless networks more than on a wired home LAN. llama.cpp accepts --ssl-key-file and --ssl-cert-file for PEM-encoded key and certificate, and hardening a local server on a LAN goes through that and the rest of the surface properly. If you are running this on a machine that holds anything sensitive, read it before you leave the configuration above running permanently.