What to Log for Every LLM Call

Almost every LLM logging setup starts as one line that writes the model name, the token counts and the duration. Every one of them is rewritten after the first incident, and the rewrite always adds the same fields. Here they are up front.

Why the SDK default is not a log

Provider SDKs give you a response object with a usage block, and it is tempting to persist exactly that. The problem is that a usage block describes an outcome and an incident is almost always about an input. Six hours later nobody asks how many output tokens were produced. They ask which prompt version was live, which model actually served the request, whether it was a retry, and what the user saw.

The second problem is that the useful fields are the ones that are only knowable at the moment of the call. Which release was deployed, which feature flag resolved which way, which of your three providers the request went to, and what the retrieved documents were. None of that can be reconstructed later from a token count, and by the time you want it, the deploy has been rolled forward twice.

The table

This is the schema the rest of this cluster writes its queries against. It is Postgres, it is deliberately wide and flat, and every column earns its place by being something you cannot derive from the others.

create table llm_request (
  request_id          uuid        primary key,
  trace_id            char(32)    not null,     -- W3C trace context
  span_id             char(16)    not null,
  parent_span_id      char(16),

  started_at          timestamptz not null,
  duration_ms         integer     not null,
  ttft_ms             integer,                  -- streaming only

  operation           text        not null,     -- chat | embeddings | execute_tool
  provider            text        not null,     -- who actually served it
  requested_model     text        not null,     -- what you asked for
  served_model        text        not null,     -- what answered (see below)

  input_tokens        integer     not null,
  output_tokens       integer     not null,
  cached_input_tokens integer     not null default 0,
  reasoning_tokens    integer     not null default 0,
  cost_usd            numeric(12,6),

  stream              boolean     not null,
  temperature         real,
  seed                bigint,
  finish_reason       text,                     -- stop | length | tool_calls | ...
  http_status         smallint,
  error_type          text,                     -- null on success
  attempt             smallint    not null default 1,

  environment         text        not null,     -- prod | staging
  release             text        not null,     -- git sha or version
  feature             text        not null,     -- see /learn/cost-attribution
  tenant_id           text,
  user_id_hash        char(64),                 -- HMAC, never the raw id
  prompt_id           text,
  prompt_version      text,

  input_hash          char(64),                 -- sha256 of the resolved body
  input_ref           text,                     -- pointer into the content store
  output_ref          text,
  provider_request_id text                      -- for escalating to the vendor
);

create index on llm_request (started_at desc);
create index on llm_request (feature, started_at desc);
create index on llm_request (tenant_id, started_at desc) where tenant_id is not null;

The fields you will wish you had

Six of those columns are the ones that are missing from most first-draft logs, and each of them corresponds to a specific bad afternoon.

• served_model alongside requested_model. If you asked for a floating alias and the provider resolved it to a new snapshot, this is the single column that shows it. Diffing the two is the cheapest possible detector for a provider-side model change.
• attempt. Retries are invisible in aggregate cost and painfully visible in p99 latency. Logging attempt number as a separate row per attempt — not one row for the whole retry loop — is what lets you compute a real retry rate and separate “slow” from “slow because we tried three times”.
• prompt_version. Without it, “quality dropped on Tuesday” has no join key. With it, the drop is a group by prompt_version.
• input_hash. A SHA-256 over the exact resolved request body. It costs nothing, it contains no personal data, and it turns “is this the same request?” into an equality test. It is also the deduplication key for building a replay corpus.
• provider_request_id. Every major provider returns an opaque id on the response — commonly in an x-request-id or vendor-prefixed header. It is the only thing a provider’s support team can act on. Not logging it means an escalation starts with “sometime around 14:00”.
• finish_reason. A truncated answer is not an error and will not appear in your error rate. It appears here, as length, and the rate of that value is one of the more reliable early signals that a prompt change has pushed outputs past their ceiling.

Where the prompt and the completion go

Not in this table. Message content has different size, different retention and different legal treatment from everything around it, so it belongs in a separate content-addressed store with its own lifecycle — object storage keyed by hash, with a TTL measured in weeks, while the metadata row above lives for a year or more.

The split buys you three things. Deleting a user’s content on request becomes a blob delete that leaves every metric intact. Retention becomes a bucket policy rather than a migration. And the expensive column is not sitting inside the table every dashboard query scans. What stays behind is input_ref and output_ref — and whether you write anything at the other end of those pointers is a decision to make deliberately, with redaction applied at capture time, not at read time.

Logs, metrics and traces are three stores

The same event wants to land in three places with different rules, and conflating them is the most common way an observability bill becomes the second-largest line item after inference.

A five-question check

Before you call a logging setup done, try to answer these five questions with a single query each. If any of them needs a code change first, the missing column is your answer.

• What did this specific bad answer cost, and which model served it?
• Which feature accounts for the largest share of yesterday’s spend?
• What fraction of requests last week were retries?
• Which prompt version was live for a request at 14:32 on the 4th?
• Which requests hit finish_reason = 'length', and is that rate rising?

A last note on what not to add. The temptation with a schema like this is to keep widening it — a column for every field the SDK returns, on the theory that storage is cheap. Resist it selectively: every column is a thing that must be populated on every code path, and a column that is null on a third of rows is worse than no column, because somebody will eventually average over it. Add a field when you can name the question it answers, and delete it when that question stops being asked.