Currency, Rounding and Why Money Should Be Integers

Usage billing for inference is an unusually hostile case for floating point: amounts far below a cent, summed millions of times, and expected to reconcile exactly against an invoice. Integers are not a style preference here.

What actually goes wrong with floats

The familiar demonstration is 0.1 + 0.2 !== 0.3, and it is usually dismissed as a curiosity worth a fraction of a cent. In a metering system it is not the representation error that hurts. It is that floating-point addition is not associative.

(a + b) + c   is not always equal to   a + (b + c)

Which means the sum of a column depends on the order the rows were added in. A database that computes SUM(cost) using parallel partial sums can return a different total on two runs over identical, unchanged data, because the partitioning differed. So can your application, if it aggregates in a different order than the reporting job did.

That is the failure that matters, and it is not a fraction of a cent — it is a system in which “the total” is not a well-defined value. Two reports disagree, nobody can reproduce either, and the engineer sent to investigate cannot find a bug because there isn’t one in the ordinary sense.

Two secondary problems come along with it. Absorption: adding a very small number to a large accumulator can change nothing at all, because the result rounds back to the accumulator — so a monthly total that has grown large stops registering individual cheap requests. And comparison: spent >= cap with floats is a comparison whose result you cannot fully reason about, which is a poor foundation for a spend cap.

Choosing the scale

The fix is to store an integer count of some small fixed unit. Which unit is not arbitrary — it follows from the smallest amount you must represent without losing information.

a typical request:  2,400 in @ $1/M  +  300 out @ $5/M
                 =  $0.0024 + $0.0015  =  $0.0039

in cents  ..........      0.39      not an integer. useless.
in micro-dollars ...  3,900         four significant figures. good.
in nano-dollars .... 3,900,000      more headroom, wider integers.

one 100-token embedding at $0.02/M = $0.000002
                                   = 2 micro-dollars
                                   = 2,000 nano-dollars

Cents are hopeless: the unit is larger than the thing being measured. Micro-dollars — 10⁻⁶ of a dollar — represent a typical request to four significant figures and an individual embedding call to one, which is the usual choice. Nano-dollars are the safer choice if you also want to store prices, because a per-token price is smaller again: $1.00 per million tokens is one micro-dollar per token exactly, but $0.15 per million is 0.15 micro-dollars and does not survive the scale.

The general rule that avoids re-deriving this: store prices at a finer scale than costs. Prices are multiplied by large token counts, so their rounding error is amplified; costs are only added. Keeping per-token prices in nano-dollars and computed costs in micro-dollars is a defensible pairing.

Range is not a constraint at these scales. A signed 64-bit integer holds about 9.22 × 10¹⁸, which is roughly $9.2 billion in nano-dollars and $9.2 trillion in micro-dollars. Neither will be your problem.

The two places integers leak

Choosing an integer type is the easy part. The value gets converted to a float at two boundaries, usually by accident.

JavaScript and JSON

A JavaScript Number is a double, so it represents integers exactly only up to Number.MAX_SAFE_INTEGER = 2⁵³−1 = 9,007,199,254,740,991.

MAX_SAFE_INTEGER in nano-dollars   = $9,007,199.25
MAX_SAFE_INTEGER in micro-dollars  = $9,007,199,254.74

A lifetime account total in nano-dollars therefore stops being exact somewhere around nine million dollars — a number a successful company reaches. In micro-dollars the ceiling is nine billion and the problem is theoretical. So: use micro-dollars in anything JSON touches, or serialise as a string and parse into BigInt. And be aware that JSON.parse silently converts an over-large integer literal to the nearest double with no error, which is the quietest possible way to corrupt an amount.

The database driver

A BIGINT column read through a driver that maps it to a language float has the same problem one layer down, and it is easy to miss because the column type looks correct. Check what your driver returns for a large BIGINT — many return a string precisely to avoid this, and code that eagerly wraps that string in a float-producing conversion undoes the protection. The same applies toNUMERIC columns, which are exact in the database and often not in the client.

Round once, at the edge

The rule is short: compute at full precision, round exactly once, at the boundary where money becomes an obligation — normally the invoice line.

Rounding each request to the nearest cent and summing them is the classic error, and its size is easy to underestimate. Rounding introduces a per-item error of up to half a cent; over a million requests, even with an unbiased rounding rule, the total drifts from the true value and, with a biased rule such as always rounding up, it drifts by up to $5,000. Sum the exact micro-dollar amounts and round the total instead.

• Pick a rounding mode and write it down. Half-up is conventional for billing and is what a customer expects. Half-even is better for statistics and will look wrong on an invoice. The important part is that every system in the chain uses the same one.
• Never round intermediate values. Not per request, not per day, not per model. Only the invoice line rounds.
• Allocate rather than round when splitting. If a total must be divided across departments or tenants, distribute the remainder deterministically — largest remainder first — so the parts sum exactly to the whole. Rounding each share independently guarantees they will not.
• Store what you rounded from. Keep the unrounded amount next to the invoiced one. Reconciliation questions are unanswerable otherwise.

A schema that survives

CREATE TABLE usage_event (
  id              BIGSERIAL PRIMARY KEY,
  request_id      UUID        NOT NULL UNIQUE,   -- idempotency
  occurred_at     TIMESTAMPTZ NOT NULL,
  api_key_id      BIGINT      NOT NULL,
  model           TEXT        NOT NULL,
  input_tokens    INTEGER     NOT NULL,
  output_tokens   INTEGER     NOT NULL,
  cached_tokens   INTEGER     NOT NULL DEFAULT 0,
  cost_micro      BIGINT      NOT NULL,          -- micro-dollars
  currency        CHAR(3)     NOT NULL DEFAULT 'USD',
  price_version   INTEGER     NOT NULL           -- which price list applied
);

• The currency travels with the amount. An amount without a currency is not an amount. Never sum across currencies without an explicit conversion step, and store the rate and the timestamp you used when you do — an exchange rate is a fact about a moment.
• The price version is not optional. Prices change. Without a record of which price list produced a cost, historical figures cannot be recomputed or defended, and a customer query about an invoice from March has no answer.
• Token counts are stored alongside the cost. They are the evidence. If a cost is ever disputed or a price was recorded wrongly, the tokens allow a recomputation; the cost alone does not.
• The request id is unique. Metering pipelines deliver at least once. Without a uniqueness constraint a retry double-bills a customer, which is a worse bug than any rounding error on this page.

None of this is specific to inference, and all of it is more load-bearing for inference than for most billing, because the amounts are small, the events are numerous and the unit price changes more often than in almost any other metered business. Get it right at the schema, once, and the rest of the cost modelling in this cluster has numbers it can trust.