“Illegal Instruction” Running llama.cpp on an Older CPU

Two words and a core dump, with no llama.cpp diagnostics at all. The absence of output is the diagnosis: the process died before it could print anything, executing a machine instruction this processor does not implement.

The string

Illegal instruction (core dumped)

On macOS the shell reports it differently, and through Python you get the same signal without a message at all — the interpreter simply disappears on import llama_cpp or on the first model load, leaving no traceback, which is why it gets misfiled as a hang or a silent crash.

Note what you did not get: no llama_model_load lines, nosystem_info: line listing the enabled instruction sets. That missing system_info: line is the single most useful signal on this page. llama.cpp prints it early, in a form like AVX = 1 | AVX2 = 1 | FMA = 1 | NEON = 0 | .... If you have ever seen it from this binary on this machine, the crash is not an instruction-set problem. If you have never seen it, it very likely is.

What an illegal instruction actually is

The kernel delivers SIGILL when the CPU decodes an opcode it does not implement. There is no recovery and no fallback: the instruction bytes are in the binary, the silicon has no circuit for them, and the process is terminated on the spot.

This happens because compilers are asked to target a specific instruction set, and llama.cpp’s default build targets the machine it is built on. The native setting reads the build host’s capabilities and emits whatever it finds. That is ideal when you build and run on the same box and actively harmful the moment those differ:

• A binary built on a modern developer laptop and copied to an older server.
• A prebuilt release or a pip wheel compiled by someone else’s newer machine. This is why the Python bindings hit it so often — almost nobody compiles those locally.
• A container image built on a newer host and run on an older one. Containers isolate userspace, not the instruction set; the CPU underneath is whatever it is.
• A virtual machine that masks CPU features from the guest. Hypervisors can present a reduced feature set for live-migration compatibility, so a guest can lack AVX2 on hardware that has it. This is a real and frequently reported cause and it is invisible unless you look.

Which instruction sets are involved

These are vector extensions, and each has a hardware generation below which it does not exist. The relevant boundaries, from Intel’s and AMD’s own documentation:

• AVX — Intel introduced it with Sandy Bridge in 2011, AMD with Bulldozer. Anything older has neither.
• AVX2 and FMA — Intel introduced these with Haswell in 2013. A large number of still-working office desktops and older Xeons predate them, which is why AVX2 is the usual culprit.
• F16C — half-precision conversion, roughly contemporary with AVX2 and often enabled alongside it.
• AVX-512 — server-first, fragmented across generations, and notably absent from several consumer parts that postdate it. A build that assumes it will fault on hardware newer than hardware that has it.

Confirming it before rebuilding

Rebuilding takes a while; checking takes ten seconds. Ask the machine what it has:

# Linux
grep -o -E 'avx[0-9a-z_]*|fma|f16c' /proc/cpuinfo | sort -u

# macOS
sysctl -a | grep machdep.cpu.features
sysctl -a | grep machdep.cpu.leaf7_features

# Windows PowerShell: report the CPU model, then check the vendor's spec page
Get-CimInstance Win32_Processor | Select-Object Name

If avx2 is absent from that output and your binary was not built for this machine, you have your cause. Run the check inside the container or VM where the crash happens, not on the host — the whole point of the VM case is that the two answers differ.

A second confirmation, if you want certainty: run the binary under a debugger and look at the faulting instruction. gdb reports SIGILL and disassembles at the program counter, and a v-prefixed AVX mnemonic there settles it.

Two false positives to rule out while you are here. A process killed for memory reports Killed, not Illegal instruction, so the two are never confusable once you read the word. And a crash with a partial llama.cpp banner — version and build info printed, then death — is more likely a bad model file than a bad instruction set, because the binary clearly executed a great deal of code before dying. The instruction-set case usually produces no llama.cpp output whatsoever, since the vector code is reached during early initialisation of the ggml backend.

Building something that runs

Turn off the native detection and then turn off each extension the target lacks. Both halves matter: disabling native without disabling the specific sets can still leave defaults enabled.

cmake -B build -DGGML_NATIVE=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF -DGGML_F16C=OFF
cmake --build build --config Release -j

On a CPU that predates AVX entirely, add -DGGML_AVX=OFF as well. For the Python bindings the same flags are passed through the build environment:

CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF -DGGML_F16C=OFF" pip install --no-binary :all: --force-reinstall llama-cpp-python

The --no-binary is not optional. Without it pip may serve you the same prebuilt wheel that crashed, and you will conclude the flags did nothing.

Expect it to be slower. Vector extensions are most of why CPU inference is viable at all, and a scalar build gives up a large multiple on the matrix work. On hardware this old, a small quantized model is the realistic target rather than a large one; the trade-offs are in choosing a quantization level.

Two neighbouring failures that are not this: a binary that dies with a message about a missing library is a linking problem, not an instruction one — see the missing libomp error — and a GPU build that fails at kernel launch rather than at startup is the wrong compute capability, which is the same class of mistake one layer down.