Logging Breaks After Switching LLM Provider: Migrating the Fields

Two shapes of this failure exist. The loud one is TypeError: Cannot read properties of undefined (reading 'finish_reason') or its Python equivalent KeyError: 'finish_reason'. The quiet one writes a log row where stop_reason is null for every request and nobody notices for a month. The quiet one is the expensive one.

The symptom

After a provider swap, some subset of the following is true: your truncation dashboard reads zero, your error-classification breakdown collapses into a single “unknown” bucket, request ids no longer join between your application logs and your trace spans, and the log line that used to tell you why a generation stopped now carries an empty field. Nothing threw, the requests succeeded, and the responses were fine. Only the observability layer is broken.

The reason it is quiet is that most logging code reads optional fields defensively. A line like finish_reason: res.choices?.[0]?.finish_reason ?? null is good practice against a malformed response and is exactly what turns a provider swap into silent data loss: the field is absent because the shape changed, the null is written, and the pipeline downstream treats null as “no value” rather than “this field is no longer being collected”. A dashboard fed by that column shows a flat line at zero, which looks like good news.

Why the fields do not line up

The mismatch is not arbitrary; it is three distinct kinds of difference, and they need different treatment.

Different names for the same concept

The simplest case. OpenAI’s Chat Completions response places the reason generation ended at choices[].finish_reason; Anthropic’s Messages API places it at the top level as stop_reason. Both are documented in the respective API references — OpenAI publishes the Chat Completions object and Anthropic publishes the Messages object. A rename is a pure mapping problem and is the easy third of this work.

Different value sets for the same field

Harder, because the mapping is not one-to-one. On Chat Completions the documented finish_reason values include stop, length, tool_calls and content_filter, plus the legacy function_call from the deprecated functions interface. On the Messages API the stop_reason values include end_turn, max_tokens, stop_sequence and tool_use. The correspondences are mostly obvious — length and max_tokens both mean the token limit was hit — but two are not. stop covers both a natural end and a custom stop sequence being matched, which the Messages API separates into end_turn and stop_sequence; and there is no direct counterpart for content_filter in that value set, because refusal is expressed differently. That is the lossy part, and the fix is not to invent a value: keep the raw provider value in its own column alongside your normalised one.

Concepts with no counterpart

Some fields simply do not exist on the other side. Request identifiers have different prefixes and different lifetimes; a determinism-related field such as a system fingerprint has no equivalent on providers that do not expose one, which is the same problem described in what to do when a provider has no seed parameter. Error shapes differ too: the HTTP status is the reliable part, and the body structure and error type strings are not portable at all.

For anything in this category, the correct move is to stop logging the provider-specific field as a first-class column and start logging a derived one you define. “Was this response truncated” is a question both providers can answer; “what was the finish_reason” is a question only one can.

Streaming makes it worse

If you log from streaming responses, the fields you need arrive at different points in the stream and the event structures are not comparable. On Chat Completions the stream is a sequence of chunk objects carrying choices[].delta, with finish_reasonpopulated on the final chunk for that choice. Usage is not included by default — you opt in with stream_options: {"include_usage": true}, after which a final chunk carries the usage object. A pipeline written without that option logs zero tokens for every streamed request, which is the same silent failure in a different place.

Anthropic’s streaming format is a set of named SSE events rather than uniform chunks: message_start carries the initial message including input token usage, then content_block_start, content_block_delta and content_block_stop per block, then message_delta — which is where the final stop_reason and the output token count arrive — and finally message_stop. Code that expects the terminal metadata on the last data frame will read it from the wrong event.

The consequence for logging is that your stream handler needs an accumulator per provider that knows which event completes which field, and it must emit the log record at stream end rather than at first chunk. The general shape of stream handling is covered in streaming transport and testing that a stream closes cleanly; what is specific to migration is that a stream aborted mid-response produces a log row with no terminal metadata at all, and you must be able to distinguish that from a provider that simply names the field differently.

The fix: a normalised log record

Define the record your dashboards consume, and make every provider adapter produce it. The rule that makes this durable is that the normalised fields are named after concepts, never after either provider’s vocabulary, and the raw provider values are preserved untouched next to them.

type LlmLogRecord = {
  // identity
  request_id: string;         // yours, generated before the call
  provider: string;           // "a" | "b"
  provider_request_id: string | null;   // theirs, whatever shape
  model: string;              // as the provider reported it back

  // outcome, normalised
  outcome: "complete" | "truncated" | "stopped_by_sequence"
         | "tool_call" | "filtered" | "error" | "client_abort";
  provider_stop_raw: string | null;  // untouched provider value

  // accounting (see the cost-dashboard page for the usage mapping)
  input_tokens: number | null;
  output_tokens: number | null;

  // timing
  ms_to_first_token: number | null;
  ms_total: number;

  // failure
  http_status: number | null;
  error_class: "rate_limit" | "timeout" | "transport"
             | "invalid_request" | "server" | null;
  provider_error_raw: string | null;
};

Two things about that record are deliberate. outcome has a value for client_abort, which neither provider reports because it is your side that stopped reading — without it, cancelled streams look like truncated responses and inflate your truncation trigger. And provider_stop_raw exists so that a value the mapping does not recognise is recorded rather than lost; the mapping function should write outcome: "complete" only for values it knows, and route anything unrecognised to a bucket that raises an alert.

Verifying the mapping

Do this before the ramp, not after, because during a ramp these fields are how you read your rollback triggers.

1. Write a test that constructs a response object of each provider’s documented shape and asserts the resulting normalised record. Fixtures captured from real calls are better than hand-written ones, since they include fields you did not know about.
2. Assert on the exhaustiveness of the stop-reason mapping: every documented value maps to a non-null outcome, and an unrecognised value produces an alert rather than a silent default.
3. Force each failure mode against both providers in a test environment — a token limit hit, a stop sequence matched, a tool call, a deliberately malformed request, a cancelled stream — and check the record each one produces.
4. Add a pipeline-level assertion that no normalised field is null for more than a small fraction of rows in any hour. This is the alert that would have caught the quiet failure in the first place, and it keeps working for every future provider change.