How Much Carbon Inference Emits: The Calculation, as a Range

The carbon cost of one AI answer is calculable, and the honest answer spans two orders of magnitude. That range is not a failure of the calculation; it is the real spread between a small model on a clean grid and a large one on a dirty grid, and any single figure you have been quoted has hidden it.

The chain of arithmetic

Four multiplications separate a request from a quantity of carbon dioxide. Every published figure is this chain with particular values chosen, whether or not the publisher shows them.

  accelerator power (W)
    x  overhead for the rest of the facility (PUE)
    /  tokens produced per second per accelerator
    =  energy per token (J)

  energy per token  x  tokens in the answer   =  energy per answer (J)
  energy per answer /  3,600,000              =  energy per answer (kWh)
  energy per answer (kWh)  x  grid intensity (gCO2e/kWh)
    =  grams of CO2 equivalent

Three of the four inputs vary by more than a factor of five in ordinary practice. That is why the output varies by two orders of magnitude, and why a point estimate is always a claim about a specific deployment rather than about AI.

Energy per token, derived

Start with the accelerator. Data-centre parts of the current generation have published thermal design powers in the region of 700 watts to a little over a kilowatt; take 700 W as the working figure and check the datasheet for the part you mean. Power usage effectiveness — the ratio of total facility power to IT power — is reported annually by the large operators and modern purpose-built facilities report figures near 1.1 to 1.2, while older mixed-use estate is worse.

  facility power per accelerator  =  700 W  *  1.2 PUE  =  840 W

Now divide by throughput. Aggregate output tokens per second per
accelerator depends overwhelmingly on model size and batching, and the
roofline derivation in the tokens-per-second page gives the ceiling.
Take a band that covers realistic serving:

  high throughput  4,000 tokens/s per accelerator
                   (small or heavily quantised model, deep batching)
  low throughput     400 tokens/s per accelerator
                   (large model, tight per-request latency target)

  energy per token, high throughput:  840 / 4000  =  0.21 J
  energy per token, low throughput:   840 /  400  =  2.10 J

In more familiar units:
  0.21 J  =  0.21 / 3600  =  5.8e-5 Wh   =  0.058 mWh per token
  2.10 J  =  2.10 / 3600  =  5.8e-4 Wh   =  0.58  mWh per token

A factor of ten, from one assumption. Note also that this is the decode side; prompt tokens are processed in parallel and cost far less energy each, which is the same asymmetry that makes prefill cheaper than decode in time and in money.

Energy per answer

Take an answer of 500 output tokens, ignoring the prompt (which adds
perhaps 10-20% at typical prompt lengths):

  low  end:  500 * 0.21 J  =    105 J  =  0.029 Wh
  high end:  500 * 2.10 J  =  1,050 J  =  0.29  Wh

A reasoning model that emits 4,000 hidden tokens before its 500 visible
ones is doing 9x the work:

  low  end:  4500 * 0.21 J  =    945 J  =  0.26 Wh
  high end:  4500 * 2.10 J  =  9,450 J  =  2.63 Wh

The reasoning-model case at the high end is 90x the simple case at the
low end, for what a user experiences as the same interaction.

That last comparison is the most important line on this page. The difference between a cheap model answering briefly and a reasoning model thinking at length is far larger than any difference between AI and its usual comparators, and it is entirely under the control of whoever chose the model. Model choice is the energy lever; reasoning tokens are where the energy goes.

Multiplying by the grid

Carbon intensity is grams of CO₂ equivalent per kilowatt hour of electricity, and it varies by more than an order of magnitude between countries and by a large factor within one country across a single day. Ember, Electricity Maps, the IEA and Our World in Data all publish it; Electricity Maps publishes it hourly by zone, which is the resolution the question actually needs.

Indicative bands, in gCO2e per kWh — check a current source for any
specific grid and hour rather than using these:

  hydro / nuclear dominated grid      tens of g/kWh
  a typical European grid average     a few hundred g/kWh
  coal dominated grid                 several hundred to ~700+ g/kWh

Applying the two extremes to our two answer estimates:

  cleanest case:  0.029 Wh = 2.9e-5 kWh  x   50 g/kWh  =  0.0015 g CO2e
  dirtiest case:  0.29  Wh = 2.9e-4 kWh  x  700 g/kWh  =  0.20   g CO2e

Range for one 500-token answer:  roughly 0.0015 g to 0.2 g CO2e.

That is a factor of 130 between two honest calculations of the same
question, and it is why a single published figure for "the carbon cost
of an AI query" tells you about the author's assumptions rather than
about AI.

Comparisons, done honestly

Comparisons are where this subject goes wrong most often, so derive them rather than quoting them.

Boiling a kettle:
  2,000 W for 3 minutes  =  2000 * 180 s  =  360,000 J  =  0.10 kWh
  Against our high-end answer at 1,050 J:  343 answers per kettle
  Against our low-end answer at   105 J:  3,430 answers per kettle

A 60 W light bulb for one hour:
  60 * 3600  =  216,000 J   ->  206 to 2,057 answers

Driving one kilometre in an average petrol car, at roughly 150 g CO2e/km:
  at 0.20 g per answer:      750 answers
  at 0.0015 g per answer:  100,000 answers

A single long-haul flight, at roughly one tonne of CO2e per passenger:
  1,000,000 g / 0.20 g  =  5 million answers

Two honest conclusions follow. One answer is a very small quantity of energy by household standards. And at a billion answers a day, the aggregate is a serious industrial load, which is why the interesting question is data-centre demand growth rather than per-query guilt — the subject of data-centre energy use.

One comparison to avoid: “an AI query uses N times a web search”. The search figure in wide circulation dates to a single company statement from 2009 and has never been updated, so the ratio is a modern number divided by a fifteen-year-old one. That trace is worked through in the unsourceable statistics page.

What this calculation leaves out

• Embodied carbon. Manufacturing the accelerator, the server and the building. No manufacturer publishes a per-accelerator life-cycle figure, so this cannot currently be derived; it is a real omission and the honest thing is to say so rather than to assume it is small.
• Training, amortised. Training happens once and is spread over every inference. Divided across a popular model’s lifetime it is small per query, but the divisor is unknown outside the provider. See training energy.
• Idle and over-provisioned capacity. Fleets are sized for peak. Electricity consumed by accelerators waiting for traffic is real and is not attributable to any query.
• Water. A separate resource with separate arithmetic, depending on the cooling design and the local climate rather than on the model. See AI water usage.
• Networking and the client device. Small relative to inference for a text answer, non-trivial for streaming media.

The published measurements

Two lines of published work are worth reading rather than re-derivations.

• Luccioni, Jernite and Strubell (2023), Power Hungry Processing: Watts Driving the Cost of AI Deployment? Measured energy per inference across a range of models and task types, and found the spread between task types — text classification against image generation — to be far larger than the spread within any one of them. If you take one finding from the literature, take that one: the task matters more than the fact that it is AI.
• Operator sustainability reports. Google, Microsoft and Amazon publish annual environmental reports with facility-level PUE and total electricity consumption. These are the only audited numbers in the whole chain, and they are for facilities rather than for queries, which is a limitation and also the reason they are trustworthy.

If you want a figure for your own application rather than for the industry, the tractable version is energy per request from your own token counts: multiply your measured output tokens by an energy-per-token figure you have chosen and stated. That is a defensible estimate. A number copied from an article is not.