Model Size Limits for WebGPU Inference in the Browser, Derived

“How big a model can I run in a browser tab” has three different answers, and the one people quote — the 256 MB buffer limit — is the one that almost never binds.

Three limits, not one

The ceilings stack, and they fail in different ways:

• Per-buffer limits from the WebGPU specification. Hard, portable, knowable in advance, and a problem for how the runtime lays out weights rather than for how large the model is.
• Total device memory the browser will let one origin allocate. This is the real ceiling, it is not in the specification, and it varies with GPU, driver and platform.
• The download and its cache. Multi-gigabyte weights over the network into storage the browser may evict. Not a hardware limit at all, and the one users actually experience.

The spec’s default buffer limits

The W3C WebGPU specification’s limits table defines default values that every conformant implementation must support. The ones that constrain an inference workload:

maxBufferSize                    268,435,456 bytes  (256 MiB)
maxStorageBufferBindingSize      134,217,728 bytes  (128 MiB)
maxUniformBufferBindingSize           65,536 bytes  ( 64 KiB)
maxBindGroups                                  4
maxStorageBuffersPerShaderStage                8
maxComputeInvocationsPerWorkgroup            256

Read those carefully, because the common misreading is that 256 MiB is the model ceiling. It is not. It is the largest single GPUBuffer you are guaranteed to be able to allocate, and a model runtime does not put a model in one buffer — it puts each weight tensor, or each shard of one, in its own. A 4 GB model spread over a few hundred buffers is entirely within the letter of these limits.

What these numbers actually constrain is the layout the runtime has to adopt. A single fused weight matrix bigger than 128 MiB cannot be bound to a compute shader in one piece and must be tiled, which is real work in the runtime and is why shipping browser inference stacks quantize and shard aggressively. You can see the constraint being honoured directly in WebLLM’s prebuilt model list, where some entries carry a buffer_size_required_bytes field — the Gemma entries request 262,144,000 bytes, 250 MB, sitting deliberately just inside the 256 MiB default.

These are defaults, not caps. The specification allows requesting higher limits through requiredLimits in requestDevice(), with the rule that requesting a capability the adapter does not support fails device creation rather than silently degrading. So asking for more is safe to attempt and must be handled when it is refused.

The limit that actually stops you

There is no specification limit on total allocation, which is exactly why it is the hard one: it is implementation-defined and differs between a discrete GPU, an integrated one sharing system RAM, and a phone. What exists instead is empirical: the requirements published by runtimes that already ship.

WebLLM’s prebuilt configuration carries a vram_required_MB figure per model, which is the maintainers’ own statement of what each needs. From that list:

Llama-3.1-8B-Instruct  q4f32_1   6,101 MB
Llama-3.1-8B-Instruct  q4f16_1   5,001 MB
Phi-3.5-mini-instruct  (range)   2,520 - 5,483 MB
Llama-3.2-1B-Instruct  q4f32_1   1,129 MB
Llama-3.2-1B-Instruct  q4f16_1     879 MB
Qwen2.5-0.5B-Instruct  q4f16_1     945 MB

Several of the smaller entries are additionally flagged low_resource_required: true while the 8B ones are not, which is the same judgement expressed as a boolean. Two things fall straight out of the table:

• An 8B model at 4-bit needs roughly 5–6 GB. That is a desktop with a discrete GPU or a machine with plenty of shared memory. It is not a mid-range phone and it is not a laptop that is also running a video call.
• Under about 1 GB is where models become broadly safe. The 1B and 0.5B entries land there, which is why browser demos converge on that size class regardless of what the developer would prefer to ship.

Note also the f16 versus f32 column: q4f16_1 costs about 18% less than q4f32_1 for the same 8B model, because the non-weight tensors and the activation path are half the width. On a device near its limit that difference decides whether the page loads, and it depends on the adapter supporting 16-bit shader arithmetic — a WebGPU feature that has to be requested and can be absent.

Add the KV cache on top of every figure above. It grows with conversation length at 2 x layers x kv_heads x head_dim x bytes_per_element per token, and it is why several of WebLLM’s entries override context_window_size down to 4096 or even 1024. A tab that works for three exchanges and dies on the twentieth is this, not a leak.

The limit users notice first

Before any of the above matters, several gigabytes have to arrive over the network. A 5 GB model on a 50 Mbps connection is about thirteen minutes, during which the page is doing nothing visible, and browsers cache that in origin-scoped storage that is subject to eviction under pressure. A user who returns next week may download it again.

This is why the practical ceiling in production is far below the technical one. A 1B-class model at roughly 900 MB is a download people tolerate once; an 8B at 6 GB is a download most people abandon, whatever their GPU could have handled. Request persistent storage explicitly, show real progress, and size the model against the connection rather than against the adapter.

Checking on the device in front of you

Every number above is a default or somebody else’s requirement. The adapter in front of your user reports its own, and the check is a few lines:

const adapter = await navigator.gpu?.requestAdapter();
if (!adapter) throw new Error("WebGPU unavailable");

console.log("maxBufferSize", adapter.limits.maxBufferSize);
console.log("maxStorageBufferBindingSize", adapter.limits.maxStorageBufferBindingSize);
console.log("shader-f16 supported", adapter.features.has("shader-f16"));

// request more than the defaults, and handle the refusal
const device = await adapter.requestDevice({
  requiredLimits: { maxBufferSize: adapter.limits.maxBufferSize },
  requiredFeatures: adapter.features.has("shader-f16") ? ["shader-f16"] : [],
});

What the adapter does not report is how much total memory you may allocate, and there is no API that will tell you. The only reliable test is allocation itself: allocate in the shards the runtime will actually use, watch for a device loss, and pick a smaller model if it comes. Build that path deliberately — device.lost is a promise, and a page that does not handle it presents an out-of-memory condition as a frozen tab.