Logs, traces and usage

What is recorded

One row per request, carrying:

• the model and provider that actually served it, and the model you asked for when those differ;
• status, HTTP status, error code and message where there was one;
• prompt, completion and cached token counts; cost and saving in micros; whether it was BYOK and whether it was a cache hit;
• time to first tokenTime to first tokenHow long you wait before the first word appears. It is what makes a model feel quick or sluggish, and it is not the same as how long the whole answer takes.Only measurable on streamed requests, which is why it is blank on some rows. and total duration, the finish reason, and how many attempts it took;
• the failover trail — every provider tried, with its status and the reason — plus attachment counts and any guardrails that fired;
• your own metadata labels, verbatim.

What is never recorded

This has a consequence worth planning for: we cannot show you the text of a request that went wrong, and neither can support. If you need the prompt, log it on your side. What we can tell you is exactly which provider saw it, when, and what it cost.

Guardrail verdicts follow the same rule. A rule that fired records its category — “payment card number” — never the value that matched it. The whole point of a PII rule is that card numbers do not end up somewhere they can be read, and the verdict is written to the request row and exported to CSV.

Labelling requests

We can tell you what you spent on a model last week. What you usually need is what you spent on the summarisation feature, or in staging, or on the one customer costing more than they pay — and only you know those, so they have to travel with the request.

{ "metadata": { "feature": "summarise", "env": "prod", "tenant": "acme" } }

Up to 16 keys, keys up to 64 characters from A-Z a-z 0-9 . _ : -, values up to 256. Numbers and booleans are coerced to strings; objects and arrays are refused rather than flattened into a structure you did not ask for. Over a limit is an error, not a truncation — a label silently cut from customer-12345 to customer-123 would merge two customers’ spend and nobody would notice.

Reading it back

Exporting to your own tracing

Configure an OTLP/HTTP endpoint under Telemetry and one span per gateway request is posted to it — Datadog, Grafana Tempo, Honeycomb, anything that speaks /v1/traces.

POST https://<your-collector>/v1/traces
Content-Type: application/json

# One span per gateway request: model, provider, attempts, token counts,
# cost in micros, TTFT and total duration. No prompt, no completion,
# no tool arguments, no system prompt.

The value is not another dashboard: it is that the time we added shows up inside the trace of the request that was waiting on us, next to your own database calls. Export is fire-and-forget and bounded by a timeout, so your collector having a bad day cannot add a millisecond to an inference call.