Sizing a Power Supply for a Local LLM GPU Rig

A supply sized to the sum of the sustained loads will run a machine that shuts down at random under inference. The number that trips protection is not the average; it is a spike lasting a hundred microseconds, and Intel publishes exactly how large it is allowed to be.

The sustained load, which is the easy half

Start with the vendor figure. NVIDIA publishes total graphics power on its product pages: 450 W for the RTX 4090, 350 W for the RTX 3090, 170 W for the 12 GB RTX 3060 (all retrieved 11 August 2026). Board-partner cards with factory overclocks are frequently above the reference figure; use the number on the box you own.

To that, add the rest of the machine. A modern desktop CPU under a light load draws well below its package power limit, but inference keeps at least one core busy feeding the GPU and, on a partial offload, all of them. Budget the CPU’s configured sustained power limit, not its idle draw. Then 20–30 W for the board and memory, 5–10 W per NVMe drive under load, and a few watts per fan.

single-card inference box, sustained

  RTX 4090 TGP                 450 W
  CPU package power limit      150 W
  board, DRAM, NVMe, fans       60 W
  --------------------------------
  sustained total              660 W

Note what is not in that sum: inference does not load the GPU and the CPU hard at the same time in the way a game does. During generation the card is memory-bound and the CPU is mostly waiting. The worst case for the supply is not steady-state inference at all — it is prompt processing on a long prompt, which is the compute-bound phase where the card actually draws its full board power.

Transients, and Intel’s published limits

A graphics card’s instantaneous draw is not its board power. The ATX 3.0 design guide sets out what a card is permitted to do in its PCIe add-in card power excursions table: for cards drawing more than 75 W, an excursion of up to 3× sustained power on the +12 V rail for up to 100 microseconds, decaying logarithmically back to 1× at one second.

Three times. A 450 W card is permitted to pull 1,350 W for a hundred microseconds, and a supply that treats that as an overcurrent condition will shut the machine down. This is the whole mechanism behind the “my PC reboots under load but the wattage looks fine” class of fault.

The corresponding obligation on the supply is in the PSU power excursion table. A supply above 450 W with a 12VHPWR connector must sustain 200% of rated power for 100 µs, 180% for 1 ms, 160% for 10 ms and 120% for 100 ms. Below 450 W and without that connector the requirements are much softer: 150% for 100 µs, 145% for 1 ms, 135% for 10 ms and 110% for 100 ms.

That difference is the practical content of “buy an ATX 3.0 supply”. It is not the connector shape; it is a written obligation to survive a doubling for a hundred microseconds. An older supply may well do it too, but nothing in its specification says it must.

Sizing the supply, derived

Two constraints, applied together. First, headroom for the transient: the supply’s rated power times its 100 µs excursion allowance must cover the sustained load plus the card’s permitted 3× spike. Second, efficiency: switching supplies peak in efficiency somewhere around half load, and running continuously near the rating shortens capacitor life through heat. Sizing so the sustained load is 50–70% of the rating handles both.

transient headroom, single 4090, ATX 3.0 supply (200% @ 100us)

  worst instantaneous  = 3 * 450 (GPU) + 210 (rest)     = 1,560 W
  supply must deliver  = rated * 2.0  >= 1,560          -> rated >= 780 W

efficiency and longevity
  sustained 660 W at 60% of rating                      -> rated >= 1,100 W
  sustained 660 W at 80% of rating                      -> rated >=   825 W

  binding constraint: the efficiency one.  850-1000 W.

NVIDIA’s own recommendation for the 4090 is a minimum 850 W system supply, which lands in the same place the arithmetic does. For the 3060, NVIDIA recommends 550 W against a 170 W card — a sustained total near 325 W, which is 59% of 550. The vendor recommendations are consistent with sizing at roughly 60% load, and that is a useful thing to know, because it means you can use them directly rather than re-deriving.

two-card layer-split box, sustained

  2 x RTX 4090                 900 W
  CPU package power limit      150 W
  board, DRAM, 2x NVMe, fans    80 W
  --------------------------------
  sustained                  1,130 W   -> at 60% of rating: 1,880 W
                                       -> at 80% of rating: 1,410 W

Which is the point at which a single-supply build stops being sensible on a domestic circuit, and where power-limiting the cards — below — stops being an optimisation and becomes the only way the machine exists.

The circuit is a real constraint and it is easy to forget, because the figures above are what the components draw and not what the machine pulls from the wall. Divide by the supply’s efficiency to get the input power: 1,130 W of load at 90% efficiency is about 1,255 W at the socket, which on a 230 V circuit is 5.5 A and on a 120 V circuit is 10.5 A — the latter being most of a standard 15 A branch that a continuous load is only rated to use 80% of. A two-card build on 120 V mains is at the edge of what one outlet is allowed to supply continuously, before anything else on that circuit.

Efficiency also varies with load, and the certification tiers are measured at 20%, 50% and 100% of rating. The curve peaks near half load, which is a second reason the 50–70% sizing target is the right one: a supply run at 90% of its rating is both hotter and less efficient than the same load on a larger unit, and the difference shows up as heat inside the case that the GPU then has to reject.

Connectors and what each is rated for

• The PCIe slot itself supplies up to 75 W to the card, from the motherboard, and every card takes some of its power that way.
• A 6-pin PCIe connector is rated 75 W; an 8-pin is 150 W. A 450 W card therefore needs three 8-pin cables, which NVIDIA states directly on the 4090 page — or one 12VHPWR/12V-2x6 cable rated for the card’s class.
• Use separate cables, not daisy chains. A single pigtailed cable with two 8-pin ends is one conductor set carrying what two should. This is the most common cause of instability in a build whose supply is nominally large enough.
• Seat 12VHPWR fully. The documented failure mode for that connector is thermal damage from partial insertion, and it is the reason to inspect the housing on any used card that has one.

Capping the card instead of buying a bigger PSU

Because single-stream generation is memory-bound, the shader cores spend most of a token step waiting. That makes a power cap unusually cheap for this workload compared with gaming: reducing the board power limit reduces core clocks, and core clocks are not what the token rate is bounded by. Memory clocks are largely unaffected by the power limit.

1. Read the current and permitted limits: nvidia-smi --query-gpu=power.limit,power.min_limit,power.max_limit --format=csv.
2. Set a cap: sudo nvidia-smi -pl 300 for a 300 W limit. On Windows this requires administrator rights and does not survive some driver updates.
3. Measure the cost with llama-bench at the same context length before and after, rather than assuming it. The generation figure and the prompt-processing figure will move by different amounts — prompt processing is compute-bound and loses more.
4. Make it persist across reboots with a systemd unit or a startup task; -pl is not sticky by itself.
5. Re-derive the supply requirement with the capped figure. A pair of 4090s at 300 W each drops the sustained total from 1,130 W to 830 W, which is a different class of supply and a different wall socket.