Reading Cost Analytics in Cloudflare AI Gateway

The gateway dashboard shows a cost line, and it is tempting to treat it as a bill. Cloudflare documents it as an estimation derived from token counts, and there are three specific situations in which it is absent or wrong. Knowing which is the difference between a useful chart and a reconciliation argument.

The five metrics

Cloudflare documents AI Gateway analytics as tracking five things: total requests, token usage across requests, costs associated with different providers, errors across the gateway, and the percentage of responses served from cache. Everything on the dashboard is a slice of those, cut by model, by provider, by gateway, and by time.

Two of the five are directly measured and reliable: request count and error count are things the gateway itself observes. Cached-response percentage is likewise the gateway’s own bookkeeping. Token usage is read out of the provider’s response body, which means it is as accurate as the provider’s own accounting — and absent when the provider does not report it. Cost is computed from tokens, and that is where the care is needed.

Cost is an estimate, and says so

Cloudflare’s costs page states that the cost metric is an estimation based on the number of tokens sent and received in requests, and directs you to your provider’s dashboard for accurate cost details. It also states that cost metrics are only available for endpoints where the models return token data and the model name in their responses.

Which produces three failure modes worth recognising on sight:

• A model with no cost line at all. A newly released model whose price is not yet in Cloudflare’s table, or an endpoint that does not return usage, shows tokens or requests but no money. The chart is not broken; the input is missing.
• Cached responses reading as free. They are free — no provider call happened — but if your hit rate is climbing, your cost chart falls faster than your traffic, and comparing the two months without accounting for that will mislead you.
• A negotiated rate that the list price does not reflect. If you have committed-use pricing, an enterprise discount, or credits, the gateway is computing at published rates and will overstate your spend consistently. That one has a fix, below.

None of this makes the number useless. Estimated cost by model and by day is exactly the right tool for finding the endpoint that quietly became 60% of spend. It is the wrong tool for closing the books.

Correcting the price with a header

For negotiated rates, Cloudflare documents the cf-aig-custom-cost request header, which takes per_token_in and per_token_out — your cost per single token, input and output. The documentation notes there is no limit on decimal places, which matters because a per-token price is a very small number and rounding it to six places is a real error at volume.

curl -X POST "$GATEWAY_URL/compat/chat/completions" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H 'cf-aig-custom-cost: {"per_token_in": 0.000001, "per_token_out": 0.000003}' \
  -d '{"model": "openai/gpt-4.1-mini", "messages": [{"role":"user","content":"hi"}]}'

Two details from the documentation are worth carrying: custom costs appear underlined in the logs so you can see which entries were adjusted, and a cache hit is recorded at zero cost regardless of the custom cost you sent. The second is correct behaviour and still surprising if you are trying to reconcile a total.

Send the header from one place in your code, next to wherever your rate card is defined, rather than sprinkling literals through call sites. A per-token price that appears in nine files is a per-token price that will be updated in eight.

Logs are the row-level view

Analytics is aggregated, and an aggregate cannot tell you which request cost forty cents. That question is answered by the log, which Cloudflare documents as containing the user prompt, the model response, the provider, a timestamp, request status, token usage, cost, duration and the client user agent — a row per request rather than a bucket per hour.

Read that field list again and notice the first two. By default the gateway stores the prompt and the completion, which for most applications means storing customer text in a system whose retention you did not choose. Two headers control this per request, and knowing the second one exists is the point of this section:

• cf-aig-collect-log — true or false, bypassing the gateway’s default logging setting for this request. Setting it to false gives you no log entry at all, and therefore no token or cost figure either.
• cf-aig-collect-log-payload — set to false, payload storage is skipped while a metadata-only log entry is still saved. You keep the usage metrics and lose the text.

// Keep the numbers, drop the customer text.
headers: {
  "cf-aig-collect-log": "true",
  "cf-aig-collect-log-payload": "false",
  "cf-aig-metadata": JSON.stringify({ tenant: tenantId }),
}

That combination is the one most production applications should be running: cost, tokens, duration and status per request, tagged with a tenant, with no prompt bodies retained. Turn payload logging on deliberately and temporarily when you are debugging a specific behaviour, rather than leaving it on because it is the default.

There is a capacity dimension too, and it is one to look up rather than assume. Cloudflare documents that each gateway has a storage limit based on your plan and points at its limits reference for the figure. Do not design around a number you half-remember: open the AI Gateway limits page, read the stored-logs figure for your plan, and divide by your request rate to find out how many days of history you actually have. If the answer is shorter than your incident-review cycle, the fix is to export what matters — which is what the GraphQL query below is for.

Querying the same data with GraphQL

The dashboard is a view; the data is queryable. Cloudflare documents a GraphQL analytics API at https://api.cloudflare.com/client/v4/graphql with the aiGatewayRequestsAdaptiveGroups dataset, whose dimensions include model, provider, gateway and a timestamp.

query GatewayUsage($accountTag: String!, $start: Time!, $end: Time!) {
  viewer {
    accounts(filter: { accountTag: $accountTag }) {
      requests: aiGatewayRequestsAdaptiveGroups(
        limit: 1000
        filter: { datetimeHour_geq: $start, datetimeHour_leq: $end }
        orderBy: [datetimeMinute_ASC]
      ) {
        count
        dimensions {
          model
          provider
          gateway
          ts: datetimeMinute
        }
      }
    }
  }
}

This is the query to reach for when you want a weekly figure in your own dashboard rather than a browser tab, or when you want an alert on a model whose request count doubled. Run it from a scheduled Worker and write the result somewhere you keep operational history — a D1 table is enough for a small team.

Splitting spend by team or user

The dimension people actually want is not model or provider — it is “which customer” or “which feature”. The gateway does not know either unless you tell it. Cloudflare’s custom-metadata documentation describes cf-aig-metadata as tagging requests with custom data such as user ids, accepting string, number and boolean values, with up to five entries per request and additional entries ignored.

headers: {
  "cf-aig-metadata": JSON.stringify({
    tenant: tenantId,      // string
    feature: "summarise",  // string
    plan_paid: isPaid,     // boolean
  }),
}

Five entries is a real constraint and worth designing around: pick the axes you will actually slice by and encode the rest into one of them rather than discovering that entries six and seven were silently dropped. Objects are not supported as values, so a nested context blob will not work.