Local LLMs for a Small Business Without a Cloud Budget

A machine that can run a good open model is a one-off cost you can put on a balance sheet, against a per-token bill that never stops. The arithmetic is genuinely favourable at high volume. It is also the least important part of the decision, and this page argues why.

The comparison as usually framed is wrong

The usual framing sets hardware capital cost against monthly API spend and finds a break-even month. That comparison is only valid if the two sides do the same job, and for most small businesses they do not.

The model you can run on a machine you can afford is not the model you are currently paying for. It may be good enough — for classification, extraction, summarising, drafting and tidying text, often it is entirely good enough — but that is a question about your task, answered by trying it, not a question about money. Deciding the money question first and then discovering the quality question is how a business ends up with a depreciating asset in a cupboard. Test the model first, on your actual work; the honest survey of what to expect is in how good local models actually are.

If it is good enough, the arithmetic below tells you what it saves. If it is not, no arithmetic helps.

The break-even, written out

Fill in your own numbers. Every symbol here is something you can look up in an invoice or a spec sheet; none of them is supplied by this page, because none of them is knowable for your business from here.

  H  = hardware capital cost, once
       (machine + GPU + storage + any UPS)

  A  = current API spend, per month

  P  = power cost, per month  (derived below)

  L  = labour cost, per month
       (hours spent operating it x loaded hourly rate)

  break-even months = H / (A - P - L)

If (A - P - L) is zero or negative, there is no break-even.
The hardware never pays for itself, however long you keep it.

Two adjustments make it honest. First, replace H with H minus residual value if you would sell the machine, and spread it over a realistic service life — three years is a common assumption for hardware you intend to keep current, and it is an assumption, not a fact. Second, A should be your spend at the volume you actually run, not at the volume you fear: teams routinely estimate API spend from a worst-case month and compare it against a best-case hardware cost.

The shape of the result is what matters more than the number. Break-even is a function of volume, and it moves fast. At low volume the hardware never pays back, because A is small and P plus L is not. At high, steady, predictable volume it pays back quickly, because A scales with tokens and the hardware does not. The interesting consequence is that the case for local hardware is strongest for the most boring workload you have — the batch job that runs every night on the same shape of document — and weakest for the exploratory, spiky, occasional use that most small businesses actually start with. That is the opposite of how it is usually pitched. The per-token side of this comparison is worked in more detail in local versus API cost.

The running cost you can actually derive

P is the one term you can compute rather than guess, because power draw and electricity price are both published numbers.

  kWh/month = (average watts / 1000) x hours per day x 30

Worked example, with every assumption stated:
  300 W average draw under a mixed load
  8 hours a day of actual use
  -> 0.3 x 8 x 30 = 72 kWh/month

  plus idle, because the machine is on all the time:
  60 W x 16 h x 30 / 1000 = 29 kWh/month

  total 101 kWh/month
  at YOUR tariff, in YOUR currency:  101 x tariff = P

Every figure above is an assumption you must replace. Use the board power your GPU vendor publishes rather than a review’s measured peak, add the rest of the system, and take the electricity price from your own bill — national averages published by statistical agencies are fine for a first pass and can be off by a factor of two against a specific tariff. Note that the idle line is often the larger one: a machine kept on so the model is warm burns power for twenty-four hours to be useful for eight.

The term that usually decides it

L is the term small businesses omit and the term that most often makes the arithmetic fail.

Running your own inference means somebody owns: patching the host, updating the runtime when a model needs a newer build, re-testing prompts when the model changes underneath them, monitoring whether the service is up, restarting it when it is not, backing up the configuration, and being the person who is called when it breaks during a busy week. In a business with no dedicated IT function that person is whoever is most technical, and their time is the most expensive resource in the company.

Put a real hourly rate against a real estimate of hours. Two hours a month is an optimistic steady state and it is not zero; the first month is many multiples of that. If the result exceeds the API bill it was meant to replace, the arithmetic is finished and the answer is no.

The second omitted cost is risk. A hosted API has redundancy you did not buy. One machine has none: if it fails, the capability is gone until it is repaired or replaced, and a replacement GPU is not a same-day item. Either accept the outage, or keep an API account as a fallback — in which case you are paying for both, and the break-even moves accordingly.

When local genuinely wins

Stripped of the enthusiasm, there are four situations where the answer is clearly local, and they are not primarily about money.

• The data may not leave. A contractual or regulatory boundary that forbids sending the data to a third party is not a cost to be optimised. It is a constraint, and local inference is one of the few ways to satisfy it — subject to everything in what local inference does not guarantee.
• High, steady, unglamorous volume. A predictable batch workload on a small model is where the per-token bill is largest relative to the difficulty of the task, and where a modest machine can genuinely absorb all of it.
• The task is narrow. Classification, extraction, redaction, tagging and reformatting are well within a small model’s reach. If that is the whole job, you are not giving anything up.
• You already have the hardware and the person. If there is a capable machine and somebody who enjoys running it, H and L are both already paid and the calculation is trivially favourable.

And the honest middle path, which is what most businesses in this position actually end up doing: run the small local model for the bulk of the volume and keep a hosted account for the cases it cannot handle. That keeps the bill proportional to the hard work rather than to all the work, and it removes the single-machine failure risk. The pattern and its trade-offs are in hybrid local and API inference.