How Much RAM You Need to Run a 70B Model on a Mac

The usual answer is “64 GB”, and it is wrong more often than it is right, because it quotes only the weights and only at one quantization. The requirement is three terms, all of which can be derived rather than guessed.

The weights, exactly

Take a specific model rather than an abstract 70B. Hugging Face reports the safetensors index for meta-llama/Llama-3.3-70B-Instruct as 70,553,706,496 parameters — a 70B is not 70e9, and the difference is eight hundred million parameters. At bf16 that is 141 GB, which is why nobody runs one unquantized on a personal machine.

MLX’s default quantization stores 4 bits per weight plus an fp16 scale and an fp16 bias per group of 64, so 4.5 bits per weight:

70,553,706,496 x 4.5 / 8  =  39,686,459,904 bytes  =  39.7 GB

Checked against the real conversion: mlx-community/Llama-3.3-70B-Instruct-4bit reports eight safetensors shards totalling 39,688,567,605 bytes through the Hugging Face model API, read on 11 August 2026. The prediction and the file agree to within about two megabytes, the difference being the layer norms that stay in fp16 and the safetensors headers.

So: 39.7 GB, and that is only term one. A machine with exactly 40 GB of usable memory does not run this model; it runs the weights and nothing else.

The context, per token

The KV cache is allocated as the conversation grows and is read in full at every decoding step. Its size per token comes straight from the model’s config.json, which for this model gives 80 hidden layers, 8 key-value heads and a head dimension of 128:

2 (K and V) x 80 layers x 8 kv_heads x 128 head_dim x 2 bytes (fp16)
  = 327,680 bytes per token
  = 0.328 MB per token

  4,096 tokens  ->   1.34 GB
  8,192 tokens  ->   2.68 GB
 16,384 tokens  ->   5.37 GB
 32,768 tokens  ->  10.74 GB
131,072 tokens  ->  42.95 GB   (larger than the weights)

Grouped-query attention is doing a lot of work here: this model has 64 attention heads but only 8 key-value heads, so the cache is one eighth the size it would be under full multi-head attention. On an older architecture without it the 32k figure would be 86 GB rather than 10.7.

Two things follow. First, “how much RAM for a 70B” has no answer until you say how much context, and the honest form of the question is 39.7 GB plus 0.328 MB per token you intend to use. Second, the cache is the term you can shrink cheaply — --kv-bits 4 quarters it, and --max-kv-size caps it outright at the cost of forgetting the start of long conversations.

What macOS lets the GPU hold

The third term is the one usually replaced by a guessed percentage. Don’t guess: Metal reports it, and MLX surfaces the report.

python - <<'PY'
import mlx.core as mx
d = mx.device_info()
gb = lambda n: round(n / 1e9, 2)
print("installed RAM        ", gb(d["memory_size"]), "GB")
print("max working set      ", gb(d["max_recommended_working_set_size"]), "GB")
print("largest one buffer   ", gb(d["max_buffer_length"]), "GB")
PY

memory_size comes from sysctl hw.memsize and is your installed RAM. max_recommended_working_set_size is Metal’s own answer to how much the GPU may hold, and it is always meaningfully below the total, because macOS, the window server and everything else you have open need the rest. That number, minus whatever you actually intend to keep running, is the real budget — and it is specific to your machine and your OS version rather than to a blog post’s seventy-five per cent.

Apple documents raising the underlying limit with sudo sysctl iogpu.wired_limit_mb=<size_in_megabytes>, which buys headroom by taking it from the operating system. It does not survive a reboot, and set too aggressively it makes the machine unresponsive rather than making the model fail cleanly. How MLX manages memory covers the allocator controls that go with it.

What it does to the rest of the machine

The three terms so far treat the model as though it were the only thing running. It is not, and this is where a figure that fits on paper stops fitting in use — because on Apple silicon there is no second pool to be displaced into.

Everything you can see is already in that pool. The window server composites every window from buffers held in the same unified memory the model wants; a browser with a real session is routinely several gigabytes and each open tab is more; a language server or a running container is several more. None of that is optional in the way a background process is optional — dropping the compositor’s working set is dropping your display.

So the honest form of the budget subtracts what you actually keep open, measured rather than assumed. Sort Activity Monitor’s Memory tab by the memory column with your normal set of applications running, and take the total. On a machine that is also your desktop, four to six gigabytes is a realistic floor and ten is not unusual. A 64 GB machine reporting, say, a 56 GB working set with 8 GB of desktop already resident has 48 GB for the model — which is 39.7 GB of weights and about 25,000 tokens of context before anything has to give.

The failure when it does give is the part worth naming, because it is a different failure from the one people arrive expecting. A discrete graphics card has a hard partition: the allocation fails, CUDA raises an out-of-memory error, your process stops and the rest of the machine carries on as though nothing happened. That is a clean, loud, local failure — and it is the behaviour behind the CUDA out-of-memory error that most local-inference advice is written against.

A Mac has no partition to fail at. Over-commit and macOS does not refuse the allocation; it starts reclaiming — compressing pages first, then paging them to the SSD. Both are system-wide, so the cost does not land on the process that caused it. Every application shares the degradation, the pointer stutters, and the model keeps answering, slowly, which is why the situation persists instead of terminating. And it compounds: decoding reads the whole model once per token, so any part of it living on storage is fetched at storage speed on every single token rather than once. The rate does not decline gently, it collapses.

The practical rule that follows is to leave more headroom than the arithmetic strictly demands. A configuration that fits with two gigabytes spare is one long pasted document away from a machine you cannot use to close the process that is doing it. The diagnosis, the Activity Monitor signals to read and the order to fix them in are in running out of memory on a Mac running a local model.

What that means per machine

Adding the terms — 39.7 GB of weights, a few gigabytes of cache for a working context, and a working-set ceiling below installed RAM:

• 32 GB or less. Not at 4 bits. The weights alone exceed the whole machine.
• 48 GB. Marginal and unpleasant. 39.7 GB of weights against a working set that will be some way under 48 leaves very little for context, and the failure mode is the machine swapping rather than a clean error.
• 64 GB. The first configuration where this is comfortable at 4 bits. Roughly twenty gigabytes above the weights, which covers a long context and leaves the desktop usable.
• 96 GB and above. Comfortable at 4 bits with a very long context, or a step up to 6 or 8 bits. An 8-bit conversion of this model is 8.5 bits per weight by the same arithmetic, which is 75.0 GB of weights — so 8 bits is a 96 GB-and-up proposition.

For which specific machines reach those tiers, and at what memory bandwidth, the Mac Studio memory ceiling page works through Apple’s published configurations.

Making it fit

If the answer is that it does not fit, the levers are these, in descending order of how much they buy:

• Fewer bits. 3-bit is 3.5 bits per weight by the same group arithmetic, or 30.9 GB — which brings a 70B into 48 GB. The quality cost is real and is largest on exactly the long-reasoning work people want a 70B for.
• Quantize the cache. --kv-bits 4 takes the 32k cache from 10.7 GB to a little under 3 GB, and unlike weight quantization it costs almost nothing at short context.
• Cap the context. --max-kv-size bounds the term entirely.
• A smaller model. Frequently the right answer. A 14B at 4 bits is 8.3 GB and leaves you a machine you can still use; whether the 70B is actually better at your task is a question for your own evaluation, not for a parameter count.