Jan.ai’s Model Stuck on Loading

The model row spins, the thread will not accept a message, and nothing in the interface ever says why. There is a message; it is in a log file two clicks away, and which of the two log files it is in narrows the cause immediately.

There is no error string, and that is the clue

Jan is a desktop application over a local inference engine that runs as a separate process. The interface asks the engine to start a model and waits for it to report readiness. Everything you can see is that wait. If the engine process dies — killed by the operating system, crashed on a malformed file, or never started at all — there is nothing to report readiness and nothing to report failure either, so the interface waits indefinitely.

That is why this is a fix page with no string to open on: the absence of a message is itself diagnostic. A model that is merely slow to load shows disk activity and finishes; a model whose engine has gone shows neither activity nor an end.

Where the message actually is

Jan writes an application log and the inference engine writes its own. Open them from the settings pane — there is a menu entry that reveals the log directory — or from a terminal:

# macOS / Linux, typical location
tail -n 200 ~/jan/logs/app.log
tail -n 200 ~/jan/logs/cortex.log

# Windows PowerShell
Get-Content "$env:USERPROFILE\jan\logs\app.log" -Tail 200

Read the engine log first. If it stops mid-load with no error, the process was killed from outside and you are in the memory case. If it reports a parse failure, a bad magic value or a truncated read, you are in the download case. If it never appears at all, the engine did not start and the problem is the backend selection rather than the model.

Cause one: the engine was killed for memory

A GGUF model has to be resident to run. The weights file is the floor, and the KV cache for whatever context length is configured sits on top of it. When the total exceeds what the machine can give, the operating system terminates the process: the Linux OOM killer, macOS jetsam, or on Windows an allocation failure that ends the process abruptly.

You can predict this rather than discovering it. The file size on disk is the weight cost almost exactly, and the KV cost per token is derivable from the model’s own configuration — the arithmetic is worked through in the slot-budget page. A 13B at Q4 is roughly a 7 to 8 GB file; running it plus a desktop plus a browser on a 16 GB machine is tight, and on a machine that also has to keep a GPU allocation resident it may not fit at all.

The tell in the logs is a load that progresses and then simply ends. On macOS you can confirm it from the system log rather than inferring it: a jetsam termination is recorded, and a diagnostic report appears in the user’s log directory. On Linux, dmesg names the killed process explicitly.

There is a compounding version of this worth knowing about. Reports on the Jan tracker describe a model staying resident after a failed load, so each retry starts with less memory than the last and fails sooner — which reads as the application getting progressively worse. If a second attempt fails faster than the first, quit the application entirely and check for a leftover engine process before trying again.

Cause two: the file on disk is not complete

The other half of these reports are a download that stopped. Jan shows a model as available once the entry exists, and an interrupted transfer can leave a file that is present, correctly named and short. The engine then loads until it runs off the end of the tensor data and stops.

Compare the size on disk to the size on the model’s own page. The publisher lists it, and a mismatch is definitive — there is no legitimate reason for a GGUF to be smaller than published:

ls -lh ~/jan/models/**/*.gguf
head -c 4 model.gguf | xxd     # expect: GGUF

A file whose first four bytes are not GGUF is a different problem again — the download brought back something that is not a model at all, which is the same class of failure covered in the invalid-file-magic page. Delete the file from within the application rather than from the filesystem, so the entry goes with it, then download again.

Two causes that look the same

• Backend selection. Jan chooses between CPU and a GPU backend, and its own troubleshooting documentation notes that a GPU is only preferred when it reports at least 6 GB of VRAM. On a machine with a small or mis-detected GPU the engine may attempt an initialisation that fails silently. Forcing CPU is a fast way to test this: if the model loads on CPU, the model is fine and the backend was the problem.
• A context length larger than the machine. The configured context is allocated at load, so a model that loads at 4,096 and hangs at 32,768 is a memory case wearing a different hat. Halve the context and try again before concluding the model is broken.

The order to work through it

1. Quit the application completely and confirm no engine process survives it. Everything below is unreliable if a previous attempt is still holding memory.
2. Open the engine log and read the last twenty lines. This decides which of the two causes you have and saves every other step.
3. Check the file: size against the publisher’s figure, first four bytes against GGUF. If either is wrong, re-download and stop here.
4. Lower the context length, then try a smaller quantization of the same model. A Q4 file is roughly half the size of a Q8 one, and that is usually the difference between fitting and not.
5. Force the CPU backend once, as a test. If it works, the model is sound and the remaining work is on the GPU configuration.