Migrating Structured Logging Schemas for LLM Calls

10 min read · updated August 11, 2026

The logging rewrite is the second half of a provider migration that nobody schedules, because the log line was written by copying whatever the SDK response object happened to contain.

Why the log is coupled to the provider

The usual call log is a serialised response object with a timestamp. That works until the response object changes shape, at which point three things break at once: the dashboards that read the old field names go flat, the cost calculation that multiplied a specific field by a specific price returns nothing, and every historical comparison spanning the cutover is meaningless because the two halves are counting different things.

Concretely, token usage is reported under different names by different APIs. OpenAI’s Chat Completions returns usage.prompt_tokens, usage.completion_tokens and usage.total_tokens; its Responses API uses usage.input_tokens and usage.output_tokens; Anthropic’s Messages API uses usage.input_tokens and usage.output_tokens with cache accounting in usage.cache_read_input_tokens and usage.cache_creation_input_tokens. A dashboard panel summing prompt_tokens reports zero the day you move, and zero renders as a healthy-looking flat line rather than an error.

The canonical record

Write your own field names and map into them at the adapter boundary. The record below is deliberately flat where it can be, because flat fields index and aggregate cheaply, and nested only where a group has to stay together.

{
  "ts": "2026-08-11T09:14:02.117Z",
  "trace_id": "...", "span_id": "...",
  "tenant_id": "...", "route": "support.reply", "role": "chat.default",

  "provider": "acme",
  "api_model": "acme-large-2026-05",
  "provider_request_id": "...",     // for support tickets and reconciliation
  "streamed": true,
  "attempt": 1,

  "tokens": {
    "input": 4120,
    "output": 318,
    "reasoning": 0,                 // null where the provider does not report it
    "cache_read": 3840,
    "cache_write": 0,
    "total": 4438                   // computed by you, never copied
  },

  "cost": {
    "input_usd": 0.0041,
    "output_usd": 0.0048,
    "total_usd": 0.0089,
    "price_version": "2026-08-01"   // which price table produced these
  },

  "latency_ms": { "ttft": 612, "total": 4310 },

  "outcome": {
    "status": "ok",                 // ok | error | cancelled
    "stop": "completed",            // canonical enum, see below
    "http_status": 200,
    "error_class": null             // your enum, not the SDK's class name
  },

  "prompt_version": "support.reply@7",
  "request_fingerprint": "sha256:..."
}

Three fields in there are the ones teams add later and wish they had added first. price_version makes a historical cost figure reproducible when a price table changes — without it, recomputing last quarter’s spend gives a different answer than last quarter did. attempt separates retries from distinct requests, without which your request count and your bill disagree and nobody can say why. provider_request_id is the only handle a provider’s support team can act on. The broader question of which fields belong in a log at all is covered in what to log, and the downstream use in cost attribution.

Mapping each provider onto it

The mapping lives in the adapter, next to the response parser, and is the only code allowed to know a provider’s field names. A rough shape:

function toCanonical(provider, res) {
  if (provider === "openai_chat") {
    const u = res.usage ?? {};
    return {
      tokens: {
        input: u.prompt_tokens ?? null,
        output: u.completion_tokens ?? null,
        reasoning: u.completion_tokens_details?.reasoning_tokens ?? null,
        cache_read: u.prompt_tokens_details?.cached_tokens ?? 0,
        cache_write: 0,
      },
      provider_request_id: res.id ?? null,
      stop: STOP_MAP.openai[res.choices?.[0]?.finish_reason] ?? "unknown",
    };
  }
  if (provider === "anthropic_messages") {
    const u = res.usage ?? {};
    return {
      tokens: {
        input: u.input_tokens ?? null,
        output: u.output_tokens ?? null,
        reasoning: null,
        cache_read: u.cache_read_input_tokens ?? 0,
        cache_write: u.cache_creation_input_tokens ?? 0,
      },
      provider_request_id: res.id ?? null,
      stop: STOP_MAP.anthropic[res.stop_reason] ?? "unknown",
    };
  }
  throw new Error("no usage mapping for provider: " + provider);
}

The canonical stop enum is worth defining explicitly rather than storing the raw string, because branching on the raw string is exactly the coupling the audit found. A workable set is completed, truncated_output, stop_sequence, tool_call, filtered, refused, unknown. OpenAI’s stop maps to completed, its length to truncated_output, its tool_calls to tool_call and its content_filter to filtered; Anthropic’s end_turn maps to completed, its max_tokens to truncated_output, its stop_sequence to stop_sequence and its tool_use to tool_call.

Make unknown loud. An unmapped value should increment a metric and log the raw string, because an unmapped stop reason is how a provider tells you it added a behaviour you do not handle — see silent model updates.

Three places the mapping loses information

Pretending the mapping is a rename is how the canonical log ends up producing confidently wrong numbers.

Cache tokens are not ordinary input tokens. Where a provider reports cache reads separately, those tokens are typically billed at a different rate from fresh input, and where it reports a cache write, that is often billed at a premium. Summing everything into one input count and multiplying by one price is wrong in both directions at once. Keep the components separate and price them separately — which is also why tokens.total should be computed by your own rule rather than copied from any provider’s total_tokens, since not every API reports one and those that do do not all include the same components.
Some stop reasons have no counterpart. A content-filter outcome on one side and a refusal expressed as ordinary text on the other are the same event to a user and different records to you. Where a provider signals refusal in the body rather than in a stop field, your adapter has to detect it, and the canonical refused value is worth keeping distinct from filtered for exactly that reason.
Streaming usage arrives late, or not at all. In a streamed response the token counts are not in the first event. Some APIs require you to ask for them — OpenAI’s Chat Completions takes stream_options with {"include_usage": true} — and Anthropic’s Messages stream carries output counts on the message_delta event near the end. If the stream is cancelled you may have no usage at all, and the canonical record needs to express “unknown” rather than defaulting to zero. Zeroes here are what make a dashboard disagree with an invoice, which is the failure examined in token count mismatch.

Keep the raw envelope, briefly

Store the unmapped response metadata alongside the canonical record for a short window — long enough to debug a mapping bug, short enough that it is not a second copy of customer content living indefinitely. Strip message content from it: what you want is the usage block, the ids and the stop field, not the text.

Give the raw column a shorter retention than the canonical table and enforce it with a partition drop rather than a policy document. This is also the field most likely to carry data into places your residency audit did not cover, so it is worth deciding deliberately rather than inheriting.

Migrating without breaking the dashboards

Add the canonical columns as nullable, alongside the existing ones. Nothing reads them yet.
Dual-write: keep emitting the old fields and start emitting the canonical ones from the same adapter call, so both are populated for the same requests.
Backfill history where the mapping is derivable from what you already stored. Where it is not — cache components you never recorded — leave them null rather than zero, and make the dashboards distinguish the two. A null means “not recorded”; a zero means “measured as none”, and conflating them silently rewrites history.
Point every dashboard, alert and cost job at the canonical fields while the old ones are still being written. Verify agreement over an overlapping window before proceeding — a discrepancy here is a mapping bug, and it is far cheaper to find now than after the old fields are gone.
Stop writing the legacy fields, wait out your longest dashboard window, then drop them.
Only now switch providers. The whole point of the sequence is that the logging change and the provider change are never in flight at the same time, because when a number moves you need to know which of the two moved it.

One property is worth designing for even if you never use a gateway: the canonical record should be produced at a single point that every service shares, not re-derived in each one. Six services each mapping usage to their own field names is six places for the cache-token error above, and the divergence shows up as a cost dashboard nobody trusts. A shared internal library achieves it; so does an LLM gateway like Multigrid, which emits one normalised usage and cost record whichever provider served the request. The requirement is the same either way — one mapping, one price table, one place to fix a bug in either.