llamafile Explained: A Model and Runtime in One Executable

A llamafile is one file that is simultaneously a Windows executable, a Unix shell script, a macOS binary, a BSD binary, and a ZIP archive with model weights inside it. That is not a packaging convention; it is a set of specific tricks in the file format, and they are worth understanding because they explain both what llamafile can do and exactly where it stops.

The problem it solves

Distributing a local model normally means distributing two things that must match: weights in some format, and a runtime built for the right operating system, the right instruction set and the right GPU stack. Most of the friction in local inference is in that matching. A user on Windows with an AMD card and a user on an Apple laptop need different binaries, and the instructions diverge at step one. That friction is a real part of the calculation in self-hosting versus calling an API: the hosted option has no distribution problem at all.

llamafile, a Mozilla project now hosted at github.com/mozilla-ai/llamafile, collapses that into a single artefact by combining llama.cpp with Cosmopolitan Libc. The claim is unusually strong — the same bytes run on macOS, Windows, Linux, FreeBSD, OpenBSD and NetBSD, on both AMD64 and ARM64 — and the mechanism is what makes it credible.

Actually Portable Executable

Executable formats disagree at the very first bytes of a file. Windows wants a PE header beginning with MZ. Linux wants an ELF header beginning with a magic byte and ELF. macOS wants Mach-O. A file can only start one way.

Cosmopolitan’s Actually Portable Executable format resolves this by being a polyglot: the file opens with bytes that are a valid MS-DOS and PE header and a valid shell script at the same time. Windows sees the PE header and executes it directly. A Unix shell, handed a file it cannot identify as a native binary, falls back to running it as a shell script — and that script is the loader, which arranges for the correct native payload inside the same file to be executed. The result is one file that every one of those systems agrees is runnable, for entirely different reasons.

This is why the documented first step on macOS, Linux and BSD is chmod +x and nothing else. There is no installer because there is nothing to install: the loader, the runtime and the weights are all in the file you downloaded.

One file, two instruction sets

Portability across operating systems is only half the problem; an ARM64 Mac and an AMD64 desktop cannot execute the same machine code. llamafile handles this by concatenating builds — the file contains compiled code for both architectures, and the loader selects the one the host can run.

There is a second layer of selection below that. Within AMD64 there are large capability differences: AVX, AVX2, AVX-512, and the corresponding NEON and dot-product extensions on ARM. A binary compiled for the newest instructions crashes on older hardware; one compiled for the oldest leaves most of a modern CPU idle. llamafile dispatches at runtime, choosing the kernel that matches the CPU it finds itself on. That is the difference between “runs everywhere” and “runs everywhere at a reasonable speed”, and it is why the project can ship one artefact rather than a matrix of them.

Where the weights live

The weights are stored inside the same file using PKZIP structure — a llamafile is a valid ZIP archive as well as a valid executable, because ZIP’s directory lives at the end of the file rather than the beginning, leaving the start free for the executable headers.

The important part is what happens at load. The weights are stored uncompressed and aligned so the runtime can memory-map them straight out of the file rather than decompressing into a separate buffer. Nothing is extracted, nothing is copied, and the operating system pages in only what inference touches. A 5 GB llamafile therefore starts in about the time it takes to open a file, not in the time it would take to unpack 5 GB. The project ships a zipalign tool for exactly this: adding weights to a bare runtime with the alignment the mapping requires.

Where the trick runs out

• Windows caps executables at 4 GB. This is a hard limit of the platform, not of llamafile, and it is the reason bigger models cannot ship as a single Windows-runnable file. The documented answer is to keep the runtime and the weights separate — run the bare llamafile executable, renamed with a .exe extension, and pass the GGUF with -m. You lose the one-file property and keep everything else.
• Linux can be configured to refuse it. The APE trick depends on how the kernel dispatches an unrecognised binary, and binfmt_misc registrations — commonly WINE’s, which claims files beginning with MZ — can intercept a llamafile and hand it to the wrong interpreter. The symptom is an error mentioning run-detectors. The documented fix is to register the APE loader explicitly with the kernel; on WSL there is an equivalent involving the interop settings.
• macOS treats it as downloaded software. Gatekeeper applies, so the first run may need explicit approval, and the documentation notes cases needing the Xcode command line tools.
• Portability is not performance parity. GPU acceleration still depends on drivers and libraries that live on the host. The single file guarantees the model will run; it does not guarantee it will use your GPU.

Those caveats are the honest boundary of the idea, and none of them undoes it. For handing a working model to somebody who should not have to learn what a quantisation is, downloading one file and running it remains the shortest path that exists.