Cost Attribution: Which Feature Is Spending Your Budget

The provider sends one number. Your application has forty places that call a model, and one of them tripled last month. Attribution is the work of making that a query rather than an archaeology project — and it is almost entirely a design problem, solved before the spend happens.

One invoice line, forty features

The naive fix is to tag calls with a string at each call site. It works for about a quarter and then decays, for reasons that are worth naming because they are the design constraints:

• Call sites move. The tag lives next to the code that calls the model, so extracting a shared helper collapses six tags into one and nobody notices until the chart flattens.
• Tags are free-form. "search", "Search" and "search-v2" become three rows on a dashboard and one argument in a meeting.
• Nested calls lose it. The summariser called by the search feature gets tagged summariser, so search looks cheap and a shared utility looks like the biggest spender in the company.

Four dimensions, chosen once

Pick a small closed set, define each precisely, and make it a type rather than a string. Four is enough for almost everyone:

Add prompt_id and prompt_version if you have a registry. Resist adding more: every dimension is a column you must fill on every path, and a dimension that is null 30% of the time is worse than absent, because it silently changes the denominator of every percentage.

Set at the edge, carried in baggage

The design that survives refactoring inverts the tagging: set the dimensions once, at the boundary where the request enters your system, and have every model call read them from context. Nested calls inherit automatically. Extracted helpers keep working. Nobody has to remember anything at a call site.

OpenTelemetry already has the mechanism — baggage, which is key/value context propagated across process boundaries via the W3C baggage header, alongside traceparent. That means the dimensions survive a hop into another service without an RPC signature change.

import { context, propagation } from "@opentelemetry/api";

// Feature is a closed union, not a string. Adding one is a code review.
export type Feature =
  | "pricing_assistant" | "doc_search" | "email_draft" | "eval_harness";

/** Called once, in the HTTP handler. Everything downstream inherits. */
export function withAttribution<T>(
  a: { feature: Feature; tenantId: string; environment: string; release: string },
  fn: () => Promise<T>,
): Promise<T> {
  const bag = propagation.createBaggage({
    "app.feature":     { value: a.feature },
    "app.tenant_id":   { value: a.tenantId },
    "app.environment": { value: a.environment },
    "app.release":     { value: a.release },
  });
  return context.with(propagation.setBaggage(context.active(), bag), fn);
}

/** Called by the LLM client. Throws in dev if attribution is missing. */
export function currentAttribution() {
  const bag = propagation.getBaggage(context.active());
  const get = (k: string) => bag?.getEntry(k)?.value;
  const feature = get("app.feature");
  if (!feature && process.env.NODE_ENV !== "production") {
    throw new Error("LLM call with no attribution context — wrap the handler");
  }
  return {
    feature:     feature ?? "unattributed",
    tenantId:    get("app.tenant_id") ?? null,
    environment: get("app.environment") ?? "unknown",
    release:     get("app.release") ?? "unknown",
  };
}

The throw in development is the load-bearing line. It converts a silent gap in your cost data into a failing test the first time somebody adds a model call outside an instrumented handler — which is the only moment when fixing it is cheap.

Background work needs the same treatment and is where attribution usually first breaks, because a queue consumer has no HTTP handler to wrap. The fix is to carry the dimensions on the job payload and restore them at the top of the worker, exactly as the HTTP boundary does. A scheduled job that belongs to no tenant still needs a feature — nightly_reindex is a real answer and unattributed is not.

Note also that baggage crosses process boundaries as a header, which means it leaves your trust boundary if you propagate it to third parties. Keep tenant identifiers out of baggage values if that is a concern, and carry an opaque internal id instead; the join back to a customer name happens in your own database, where it belongs.

The queries

Against the llm_request table from the logging page, the interesting questions are short. Note that all three exclude non-production environments explicitly, because forgetting to is how an eval run gets blamed on a customer.

-- Spend by feature, week over week, with the delta that matters.
with weekly as (
  select feature,
         date_trunc('week', started_at) as wk,
         sum(cost_usd)                  as spend,
         count(*)                       as requests
  from llm_request
  where environment = 'prod'
    and started_at >= now() - interval '8 weeks'
  group by 1, 2
)
select feature, wk, spend, requests,
       round(spend / nullif(requests, 0), 6)               as cost_per_request,
       round(spend - lag(spend) over w, 2)                 as delta_usd,
       round(100 * (spend / nullif(lag(spend) over w, 0) - 1), 1) as delta_pct
from weekly
window w as (partition by feature order by wk)
order by feature, wk;

-- Where did a jump come from: more requests, or dearer requests?
select feature, release,
       count(*)                              as requests,
       round(avg(input_tokens))              as avg_in,
       round(avg(output_tokens))             as avg_out,
       round(avg(cost_usd), 6)               as avg_cost,
       round(sum(cost_usd), 2)               as spend
from llm_request
where environment = 'prod'
  and feature = $1
  and started_at >= now() - interval '14 days'
group by 1, 2
order by spend desc;

-- Waste: spend on requests that produced nothing useful.
select feature,
       round(sum(cost_usd) filter (where error_type is not null), 2)      as failed,
       round(sum(cost_usd) filter (where attempt > 1), 2)                 as retries,
       round(sum(cost_usd) filter (where finish_reason = 'length'), 2)    as truncated,
       round(sum(cost_usd), 2)                                           as total
from llm_request
where environment = 'prod' and started_at >= now() - interval '30 days'
group by 1
order by total desc;

The second query is the one that earns its keep. A cost increase has exactly two causes — volume or unit cost — and they have entirely different owners. Splitting by release in the same breath usually names the deploy.

The third is the one that changes behaviour. Waste — spend on requests that failed, were retried, or were truncated before they said anything useful — is invisible in a total and often larger than anyone expects. It is also the only category of spend that can be reduced with no product trade-off whatsoever, which makes it the first thing to look at when someone asks for a cost reduction and the answer is expected to be “use a worse model”.

Attribution coverage is itself a metric

Attribution decays quietly. The defence is to measure the decay directly: track the share of spend that landed in the unattributed bucket and alert when it crosses a threshold you have chosen — a percent or two is a reasonable place to start, and the number matters less than the fact that someone sees it move.

select date_trunc('day', started_at) as day,
       round(100.0 * sum(cost_usd) filter (where feature = 'unattributed')
             / nullif(sum(cost_usd), 0), 2) as unattributed_pct
from llm_request
where environment = 'prod' and started_at >= now() - interval '30 days'
group by 1 order by 1;

Do the same for the reverse direction: sum your computed cost_usd for a month and compare it to the provider’s actual invoice. A persistent gap means your pricing table is stale, you are missing a request path, or someone is calling the provider with a key that does not go through your client. All three are worth knowing and none of them show up any other way.

Surviving a refactor

• Make feature a type. A union or enum means deleting a feature is a compile error, not a chart that quietly stops updating.
• Never read attribution from the call stack. Inferring the feature from the calling module is clever and breaks the first time someone extracts a function.
• One place makes model calls. If your codebase has one client wrapper, attribution is enforced in one file. If it has eleven direct SDK imports, it is enforced nowhere.
• Keep dimensions out of metric labels. tenant_id on a Prometheus label is a cardinality incident. Dimensions belong on spans and in the request log; metrics get feature and environment at most.