Rewriting a LangChain Prompt Template for a Different Provider

You changed one line — the chat model the chain is built on — and a prompt template that has been in production for months now raises an exception before a single token is generated. Two different mechanisms produce that symptom and they need different fixes.

The errors you actually get

The three strings worth recognising, because they come from three different layers:

• ValueError: Missing some input keys: {'context'} — or a bare KeyError naming a variable you are certain you passed. This is LangChain comparing the variables it extracted from the template string against the keys you supplied at invoke time.
• ValueError: Single '}' encountered in format string — this one is not LangChain at all. It is Python’s own str.format, surfacing through the default template format.
• A 400 from the provider mentioning message roles or ordering, raised after the template rendered perfectly well. This is the second failure mode and the template is not the problem.

The reason all three appear on the day you swap providers is that a provider swap in LangChain usually comes with a prompt swap: somebody pastes in the few-shot examples from the new vendor’s cookbook, or converts a single-string prompt into a message list to get a system message where the new provider wants it. The template class did not change. What it is being asked to render did.

Failure one: braces in the template

A LangChain PromptTemplate defaults to the f-string template format, which means every brace in your template text is a formatting instruction. That is fine until the template contains JSON — a few-shot example of the output you want, a schema you are describing in prose, a tool call you are demonstrating. Then every brace in that JSON is read as the start of a variable name.

Two things happen. Braces that enclose something resembling an identifier become phantom input variables, and LangChain dutifully reports them missing at invoke time — which is where Missing some input keys comes from and why the named key looks nonsensical. Braces that do not pair up produce the raw str.format error about a single brace instead.

There are two fixes and they are not equivalent. The mechanical one is to double every literal brace, so {"role": "user"} becomes {{"role": "user"}}. That works and it is unreadable, and it fails the next time somebody edits the example and forgets. The better fix is to change the template format so that braces are not special: construct the template with template_format="mustache" and write placeholders as double braces in mustache style, leaving literal JSON untouched. Whichever you choose, choose it for the whole repository, because a codebase where half the templates escape braces and half do not is a codebase with a latent formatting bug in it.

One more thing to check while you are here: a template loaded from a file, a database or a prompt registry has the same problem, and it fails at render time rather than at import time. If you store prompts outside the code, the escaping convention has to be stored with them.

Failure two: the message list is not portable

The second family of failures has nothing to do with formatting. It is that the rendered message list is legal on one provider and illegal on another, and LangChain does not rewrite it for you beyond a small amount of normalisation.

The specific rules that bite. Anthropic’s Messages API takes the system prompt as a top-level system parameter, not as an entry in messages — the LangChain integration lifts a leading system message out for you, but a system message that is not first, or a second system message halfway down a few-shot sequence, has nowhere to go. The same API requires the conversation to begin with a user turn and to alternate roles; a template that emits two consecutive human messages, which the OpenAI shape accepts without complaint, is rejected. And an empty message — easy to produce when a MessagesPlaceholder renders an empty history into a turn that is otherwise blank — is rejected where it was previously ignored.

The general principle is worth stating because it outlives the specific rules: the OpenAI chat shape is permissive about message sequences and most other shapes are stricter. Migrating away from it therefore surfaces sloppiness that was never punished. That is not a reason to stay; it is a reason to render your prompt once and look at it before you debug anything else.

# Render the messages without calling a model. This is the whole
# diagnostic, and it takes ten seconds.
messages = prompt.format_messages(question="…", chat_history=[])
for m in messages:
    print(type(m).__name__, repr(m.content[:80]))

Reading that output answers, in order: did the template render at all; is there exactly one system message and is it first; do the roles alternate; is any message empty; and is there stray whitespace at the end. Four of the five failure modes on this page are visible in it.

Failure three: the last message

A template that ends with an assistant turn is doing prefill — seeding the model’s reply so it continues rather than starts. This is a real technique and it is where providers differ most sharply. Anthropic’s API accepts a trailing assistant message and continues from it, but rejects one whose content ends in trailing whitespace. Templates produce trailing whitespace constantly, because a triple-quoted Python string that ends with a newline before the closing quotes has a newline in it.

The fix is one call — strip the final assistant message’s content before sending — but the reason to know the mechanism is that the same template is fine on a provider that ignores the trailing newline, so this failure appears to be caused by the provider swap when it was latent all along. If your chain does prefill, normalise the final message explicitly rather than relying on the template author to have been careful.

The mirror image is worth a sentence: not every provider supports prefill at all. Where it is unsupported, a trailing assistant message is either an error or, worse, silently treated as an ordinary history turn — and your carefully engineered continuation becomes a stray message the model responds to. See the general treatment of capability gaps for how to decide whether to shim or drop a technique like this.

The rewrite

1. Replace the single-string PromptTemplate with a ChatPromptTemplate built from an explicit message list. A string template piped into a chat model becomes one human message, which is the shape that causes the system-message problems above.
2. Put the system content in exactly one leading system message. If you had two, merge them; if you had one in the middle, decide whether it was really an instruction (merge it up) or an example (make it a human/assistant pair).
3. Use MessagesPlaceholder for history rather than interpolating a formatted transcript into a string. A placeholder renders to zero messages when the history is empty, where a string interpolation renders to an empty turn.
4. Fix the braces once, by choosing a template format for the codebase and converting the templates that need it. Add a test that renders every template in the repository with dummy variables, so an unescaped brace fails in CI rather than in production.
5. Strip trailing whitespace from the final message if you use prefill, and assert the message list satisfies the new provider’s ordering rules before it is sent.
6. Re-run your prompt regression suite. The template now renders; that says nothing about whether the new model reads it the same way.

The last step is the one people skip. A prompt that renders is not a prompt that works, and the failure you cannot see is the one where the new model reads the same instructions and complies less exactly. Treat the rendering fixes as the precondition for the evaluation, not as the migration.