It Works in the Playground But Not in My Code

A prompt that works in a provider’s playground and fails in your code is almost never a model difference. It is a request difference: the UI supplied parameters, history, tools and a system message that your code did not, and every one of those changes the output.

The model is the same; the request is not

A playground is an application built on the same API you are calling. It constructs a request from what you typed plus a set of defaults its authors chose, and those defaults are not the ones your SDK chooses. The debugging question is therefore not “why is the model different” but “what is in their request that is not in mine”, which is answerable rather than mysterious.

The seven things a playground supplies

1. A system message. Either one the UI prefills, one the product wraps around your input, or simply the empty one you left in the box while your code sends something else. This is the single largest source of divergence, because a system message changes tone, length, format and refusal behaviour at once.
2. Sampling parameters. Temperature, top-p and penalties have UI defaults that need not match your SDK’s. A UI at temperature 0.7 and code at temperature 1.0 produce visibly different consistency, and the difference is easy to attribute to the wrong thing.
3. max_tokens. UIs typically allow a generous output. A library default, or a value you set for cost reasons and forgot, truncates the answer — which reads as a worse answer rather than as a truncated one. Check finish_reason before concluding anything about quality; truncated output covers the tells.
4. Conversation history. The playground keeps the whole thread. If you tested across four turns and your code sends one message, the model has lost every clarification those turns contained. This is the second-largest source and it is invisible in a screenshot.
5. Enabled capabilities. Toggles for web search, code execution, file retrieval, structured output or a JSON response format. A prompt that works because the UI silently retrieved a file you uploaded will not work in code that retrieves nothing.
6. A different model behind a friendly name. The UI may show a product name that maps to a current alias, while your code pins a snapshot — or the reverse. Compare the model field the API returns, not the label in the dropdown.
7. Rendering. The UI renders markdown, so tables, headings and code blocks look formatted. The API returns the same characters, and your terminal or your HTML shows them raw. The output is identical; only the presentation differs, and a surprising number of “the API output is worse” reports are exactly this.

Two more, less common but worth ruling out: safety configuration can differ between the console surface and the API surface for the same account, and some UIs apply a seed or serve a repeated identical request from a cache, making the UI look more deterministic than it is.

Capturing the request the UI actually sends

Stop inferring and read it. In descending order of reliability:

• The UI’s own export. Many playgrounds have a view-code or export affordance that emits the request as an SDK call. Where it exists, this is the authoritative answer and takes ten seconds.
• The browser network tab. Open developer tools, submit the prompt, find the request to the completions endpoint and read the JSON body. Every parameter the UI set is right there. This works even when no export exists.
• Ask the model to repeat its instructions. An unreliable last resort, and the result may be a plausible reconstruction rather than the actual system prompt. Treat what it says as a hint, not as evidence.

Then diff that body against the one your code sends. Log your outgoing request body once, in full, and compare field by field — including the fields that are present in one and absent in the other, which is where the answer usually is.

A diff harness that finds the culprit

If you cannot capture the UI’s request, work from the other end: start from a fully-specified minimal request that works, and add your application’s parameters one at a time until it breaks.

BASE = dict(
    model=MODEL,
    messages=[{"role": "user", "content": PROMPT}],
    temperature=1.0,
    max_tokens=2048,
)

VARIANTS = {
    "base":              {},
    "+ your system":     {"messages": [{"role": "system", "content": SYSTEM},
                                       {"role": "user",   "content": PROMPT}]},
    "+ your temp":       {"temperature": APP_TEMPERATURE},
    "+ your max_tokens": {"max_tokens": APP_MAX_TOKENS},
    "+ your tools":      {"tools": APP_TOOLS},
    "+ json mode":       {"response_format": {"type": "json_object"}},
    "+ full history":    {"messages": APP_MESSAGES},
}

for name, override in VARIANTS.items():
    r = client.chat.completions.create(**{**BASE, **override})
    c = r.choices[0]
    out = (c.message.content or "")
    print(f"{name:20} model={r.model:28} finish={c.finish_reason:10} "
          f"len={len(out):5}  {out[:60]!r}")

Read the table it prints rather than the outputs. The row where the behaviour changes names the parameter, and the model column catches the alias case at the same time. This takes two minutes and replaces an argument about which surface is right.

The surfaces that keep state your code does not

Some playgrounds are thin wrappers over a stateless completions endpoint. Others sit on top of a stateful assistant or thread abstraction, and those hold things your code has no equivalent of. When the difference survives every parameter check above, this is usually why.

• Uploaded files. A document attached in the UI is chunked, indexed and retrieved on your behalf. Your code gets none of that unless you built retrieval yourself, so the prompt is answering from a context that does not exist on your side.
• Thread history the UI manages. A stateful thread accumulates every turn, and it may summarise or truncate older ones under a policy you cannot see. Two sessions of the same length can therefore differ.
• Built-in tools. Web search, code execution and similar are toggles in the UI and separate integrations in code. An answer containing current facts, or arithmetic that is exactly right, is a strong hint that a tool ran.
• Account-level instructions. Consumer chat products in particular apply saved preferences and memories from the account, which are invisible in the conversation and absent from the API. This is why a prompt refined in a consumer chat app often behaves differently the first time it is called programmatically.

The practical consequence is that a prompt developed in a rich surface has been developed against a system, not against a model. Rebuild the missing pieces explicitly — your own retrieval, your own history policy, your own tools — or develop the prompt against the plain completions endpoint from the start, which is slower and produces a prompt that actually transfers.

When neither is wrong and both are different

Sometimes the requests are identical and the outputs still differ. At any temperature above 0 the model returns a sample from a distribution, so two runs of the same request are supposed to differ, and one playground run is one sample. Comparing one sample against one sample cannot distinguish a real difference from ordinary variance.

Run each side ten times before concluding anything. If the playground produces your desired output three times in ten and your code produces it two times in ten, nothing is broken and the prompt is simply not reliable enough — which is a different and more useful problem to have identified. And note that temperature 0 is not a guarantee of identical output either, for reasons covered in why temperature 0 is not deterministic. If the difference appeared suddenly rather than being there all along, nothing changed and the output changed is the better page.