Hidden Reasoning Tokens: Billed, Invisible, and Yours to Handle

A vendor table of who shows what would be wrong within a quarter. The three decisions underneath it have not changed since reasoning models shipped, and once you can see them separately, a new provider takes about ten minutes to characterise.

Three independent choices

Providers make these one at a time, and every combination exists in the wild.

• Is the trace returned? Three answers: verbatim, as a model-written summary, or not at all. Anthropic’s extended thinking returns thinking content blocks; DeepSeek’s API returns a reasoning_content field alongside content; OpenAI’s reasoning models return a summary rather than the raw trace.
• Is it billed? In practice always, at the output rate, whether or not you can read it. This is the choice with no variance and the one people are most surprised by.
• Does it persist between turns? Some APIs require you to send the previous trace back — verbatim, or as an opaque encrypted item — for the next turn to work correctly. Others explicitly forbid it and error if you try.

The third is the one that breaks code, because it is invisible until you build a multi-turn or tool-calling loop, at which point it becomes the entire problem.

Why anyone hides it

Two stated reasons, and it is worth knowing both because they pull in the same direction on visibility and opposite directions on trust.

The first is competitive. Raw traces are training data: a long, high-quality reasoning trajectory from a frontier model is exactly the input a distillation run wants, as the results in reasoning distillation demonstrate. Publishing them for the price of an API call is publishing the recipe.

The second is stated in OpenAI’s September 2024 write-up of o1 and is more interesting: they argued that leaving the raw chain of thought unconstrained — not trained to be polite, safe or policy-compliant — makes it more useful as a monitoring surface, and that showing it to users would create pressure to sanitise it. So the trace stays honest and stays hidden, and users see a summary. Whether you find that convincing, it is a real argument and not merely a commercial one.

Telling which one you are on

Send one request that provokes thinking and inspect the raw response body, not the SDK’s convenience accessor — which usually concatenates only the visible text and will hide exactly what you are looking for.

// 1. Is there a reasoning token count at all?
usage.completion_tokens_details?.reasoning_tokens   // OpenAI shape
usage.output_tokens                                 // Anthropic: thinking
                                                    //   blocks are included

// 2. Is any trace text present?
content.find(b => b.type === "thinking")            // Anthropic
message.reasoning_content                           // DeepSeek shape
output.filter(i => i.type === "reasoning")          // OpenAI Responses

// 3. Is the visible answer much shorter than what you paid for?
//    reasoning_tokens / completion_tokens > 0.5 is routine, not a bug.

If a count is present and text is not, you are billed and blind. If text is present, check whether it reads as a trace or as prose about a trace — a summary says “I considered two approaches”, a raw trace contains false starts, arithmetic and abandoned branches. The difference matters for debugging: a summary tells you what the model says it did, which is not the same thing, and that gap is the subject of the faithfulness literature.

The multi-turn problem

Here is where it stops being a curiosity. When a reasoning model calls a tool, the conversation resumes after the tool result — and the model needs its own prior thinking to continue coherently. Providers have solved this in incompatible ways.

Anthropic returns a cryptographic signature on each thinking block and requires that the block be passed back unmodified when you continue a tool-use turn; tamper with the text and the request is rejected. OpenAI’s Responses API keeps reasoning items server side when you let it, and offers an encrypted reasoning payload for callers who do not want state stored — you hold an opaque blob and hand it back. DeepSeek goes the other way: its documentation states that reasoning_content must not be included in the next request’s messages.

The consequence for anyone writing a gateway, an agent framework or merely a wrapper: a conversation history is no longer a portable list of role-and-content pairs. It carries provider-specific opaque state, and “replay this conversation against a different model” silently stops being a supported operation. Design for it now — keep the provider-native blocks alongside your own normalised history rather than trying to round-trip one into the other.

The failure mode when you get this wrong is worth recognising, because it does not look like a state bug. The model completes the tool call and then behaves as though it has forgotten why it made it: it repeats a step, re-reads a file it already read, or produces an answer that ignores what the tool returned. Nothing errors. If a tool-using reasoning agent seems oddly forgetful mid-task, check what you are sending back before you start rewriting prompts.

What being blind actually costs

• You cannot audit a decision. In a regulated workflow, “the model considered the applicant’s history” is a summary you were handed, not evidence.
• You cannot cache the expensive part. Prompt caching covers the prefix you sent. The trace is generated fresh every time, so the dearest tokens in the request are exactly the ones no cache touches.
• Cost forecasting gets a fat tail. Visible answers have a predictable length distribution. Hidden traces do not — the same prompt can produce 400 or 14,000 thinking tokens depending on whether the model found the problem interesting. Budget on p95, not on the mean.
• The context window pays too. In a multi-turn conversation where traces are carried forward, they occupy context alongside the visible history. A dialogue that would have fitted comfortably can run out of room several turns earlier than expected, and the symptom — a truncation error deep in a working conversation — does not obviously point at the cause.