Picking a GPU by VRAM per Dollar for Local Inference

The one thing a card cannot compensate for is not having enough memory, which makes dollars per gigabyte the honest first filter. It is also a ratio that gets two specific cases badly wrong, and both are worth knowing before you use it.

Why the ratio is the right first filter

Every other specification degrades gracefully. Half the bandwidth is half the token rate, which is slower but works. Half the shader cores is slower prompt processing, which is slower but works. Insufficient memory does not degrade — the model either loads or it does not, and the alternative is the partial-offload cliff derived on the 12GB page, which costs far more than any bandwidth difference between cards in the same price bracket.

So the sensible procedure is: derive the capacity your workload needs using the arithmetic on the 24GB page, discard every card below it, and only then optimise. Dollars per gigabyte is how you compare what survives the filter.

Doing it in the other order is the common mistake, and it is expensive in a specific way. Somebody picks a card on a price-to-performance chart built from gaming benchmarks, discovers the model they wanted needs two more gigabytes than it has, and then either drops a quant level they did not want to drop or runs across the offload cliff. Both of those cost more than the difference between the card they bought and the one a capacity-first filter would have chosen. Gaming charts rank on frame rate, which is a compute-and-bandwidth measure with capacity barely in it; this workload ranks almost the opposite way.

Dollars per gigabyte at announced prices

Memory sizes come from NVIDIA’s own GeForce comparison page (retrieved 11 August 2026), which lists 32 GB for the RTX 5090, 16 GB for the 5080 and 5070 Ti, 12 GB for the 5070, and 24 GB for the RTX 4090. Prices are the starting prices announced in NVIDIA’s own press releases: the RTX 50 series announcement of 6 January 2025 and the RTX 40 series announcement of 20 September 2022.

$/GB = announced starting price / published memory size

  RTX 5090    $1,999 / 32 GB  = $62.47 /GB   (announced 2025-01-06)
  RTX 5080    $  999 / 16 GB  = $62.44 /GB   (announced 2025-01-06)
  RTX 5070 Ti $  749 / 16 GB  = $46.81 /GB   (announced 2025-01-06)
  RTX 5070    $  549 / 12 GB  = $45.75 /GB   (announced 2025-01-06)
  RTX 4090    $1,599 / 24 GB  = $66.63 /GB   (announced 2022-09-20)
  RTX 4080    $1,199 / 16 GB  = $74.94 /GB   (announced 2022-09-20)

The shape is the useful part and it is consistent across both generations: the ratio is worst at the top of the stack and best in the upper-middle. The 5070 Ti and 5070 are around $46/GB while the 5090 and 5080 are around $62/GB, and the 4080’s $75/GB was worse than the 4090’s $67/GB despite costing much less — a flagship is sometimes better value per gigabyte than the tier below it, which is not the intuition people bring.

The second axis: bandwidth per dollar

Capacity decides whether a model runs; bandwidth decides how fast, and the two are correlated but not identical, because bandwidth is bus width times data rate and capacity is bus width times chip density. Computing both ratios separates cards that look alike on one.

bandwidth = (bus_width_bits / 8) * per-pin data rate

  RTX 4090: 384-bit @ 21 Gbps GDDR6X = 1,008 GB/s
            $1,599 / 1,008 = $1.59 per GB/s

  RTX 3060: 192-bit @ 15 Gbps GDDR6  =   360 GB/s
            $  329 /   360 = $0.91 per GB/s

Bus widths are NVIDIA’s published figures; the per-pin data rates are the memory grades those cards ship with and should be checked against the board partner’s sheet, since factory-overclocked memory changes the result. The 3060 figure uses its own launch price.

The reason the two ratios can diverge sharply, rather than tracking each other, is that memory capacity and memory bandwidth are set by different things. Bandwidth is bus width times the per-pin data rate. Capacity is bus width times the density of the memory chips attached to it — and a manufacturer can double capacity without touching bandwidth by mounting chips on both sides of the board in a clamshell arrangement, two chips sharing each 32-bit channel. That is how a 128-bit card ships in both an 8 GB and a 16 GB version with identical bandwidth.

For inference that arrangement is a genuinely good deal and an underrated one, because the doubled capacity is the specification with no substitute while the unchanged bandwidth only costs proportional speed. It is also why the two versions of such a card can differ by fifty per cent in $/GB and not at all in $/(GB/s), and why quoting one ratio without the other is how people end up recommending the wrong half of a pair.

Reading the two ratios together is what makes them useful. A card with good $/GB and poor $/(GB/s) will run large models slowly — fine for batch work, painful for anything interactive. A card with the reverse will be fast at whatever it can hold and unable to hold much. Which failure you can tolerate is a property of your workload, not of the card.

What the ratio hides

• Two cards are not one card. Two 12 GB cards give 24 GB of weight space at a much better $/GB than one 24 GB card, and they work — see the two-card page for why layer split is cheap. But they also need two slots, a much larger supply, and they do not give you 24 GB of contiguous space for a single tensor. The ratio says nothing about any of that.
• Running cost is not in the ratio. A 450 W card held busy for hours a day is a meaningful electricity bill, and a 170 W card doing the same work more slowly may cost less in total. Multiply board power by your hours and your tariff before treating the purchase price as the price.
• Software support has a cliff. A cheap card outside the current driver and toolkit support list is worth less than its $/GB suggests, because the support window is the useful lifetime. That check is on the used GPU page and applies to new cards from smaller vendors too.
• Unified memory changes the units. On a system where the GPU draws from system RAM, “VRAM” is a configurable share of a much larger pool and the ratio is not comparable at all.
• Second-hand inverts the table. The best $/GB available is almost always an older 24 GB card, and the reason it does not appear above is that there is no published price to cite for it.

Computing it for today’s prices

1. Derive the capacity you need first, from the model you intend to run and the context you intend to use, with the formula on the 24GB page. Add a gigabyte of margin.
2. List every card at or above that capacity, from any vendor, new and used.
3. For each, take the price you can actually pay today and the memory size from the manufacturer’s specification page, and divide.
4. Compute the second ratio too: bus width divided by eight, times the per-pin data rate, divided by the price.
5. Discard anything not on the current driver and toolkit support list, whatever its ratio, then add the power supply cost from the PSU page to any card that needs one you do not have. That is often the line that changes the ranking.