Moving From LangChain’s OpenAI Wrapper to the Native SDK

The rewrite of the call itself takes ten minutes. The reason it takes a week is everything the wrapper was quietly doing around the call, and the honest way to decide is to list those things first.

The call, both ways

Here is the same interaction expressed through the LangChain chat model and through the provider SDK directly.

# through the wrapper
from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage

llm = ChatOpenAI(model="gpt-4o", temperature=0, max_retries=3, timeout=30)
result = llm.invoke([
    SystemMessage("You are a concise support assistant."),
    HumanMessage("Where is my order?"),
])
text = result.content

# direct
from openai import OpenAI

client = OpenAI(max_retries=3, timeout=30.0)
resp = client.chat.completions.create(
    model="gpt-4o",
    temperature=0,
    messages=[
        {"role": "system", "content": "You are a concise support assistant."},
        {"role": "user", "content": "Where is my order?"},
    ],
)
text = resp.choices[0].message.content

Two lines longer, and it makes three things visible that the wrapper hid: the role strings, the shape of the response, and the fact that content can be None — which it is whenever the model returned a tool call instead of prose, and which is the first crash of every migration like this.

If your old code is much older than this, you may be on the previous major of the Python SDK, whose removal produces module 'openai' has no attribute 'ChatCompletion'. That is a separate migration — module functions became methods on a client object — and it is worth doing as its own commit before you touch LangChain.

What the wrapper was doing for you

• Message conversion. Your codebase probably passes message objects around. Every one of those has to become a dict with a role and a content, and the mapping is not one line once tool messages are involved: a tool result is a message with role tool carrying a tool_call_id, and an assistant turn that called tools has a tool_calls array and null content.
• Retries. Both the LangChain wrapper and the OpenAI SDK have their own retry configuration, and the SDK’s covers connection errors and the retryable status codes with backoff. This is the one item on this list that costs nothing to replace, because the provider SDK already does it — but check the default retry count rather than assuming it matches what you had configured.
• Timeouts. Same story, with one trap: a streaming call needs a different timeout policy from a unary one, because the meaningful failure is a stalled stream rather than a slow response. A single request timeout will either kill long legitimate generations or never fire.
• Callbacks and tracing. If you had a tracing integration attached at the wrapper level, every call site loses it at once. Decide where the replacement span is opened before you start deleting, or you will do the migration twice.
• Provider abstraction. The wrapper is the reason you could change one line to point at a different vendor. Going direct means the vendor’s shape is now spread across every call site unless you put your own thin interface in its place — which is the main argument for wrapping the direct call in one internal function rather than inlining it everywhere.

Nothing on that list is hard. The point of writing it down is that people migrate expecting to delete a dependency and discover they are writing a small client library, and a small client library that nobody decided to write is one that ends up different in each service.

Structured output is the biggest single piece

If your chain uses with_structured_output, that one call is doing several things: it turns your Pydantic model into a JSON Schema, attaches it to the request in whichever way the provider supports, parses the response, validates it against the model, and hands back a typed object. Reimplementing it means doing all five explicitly.

from pydantic import BaseModel

class OrderStatus(BaseModel):
    order_id: str
    status: str
    eta_days: int

resp = client.chat.completions.parse(
    model="gpt-4o",
    messages=[...],
    response_format=OrderStatus,
)
parsed = resp.choices[0].message.parsed   # None if the model refused

The current OpenAI SDK offers a parsing helper that takes the model class directly, which covers most of the gap. What it does not cover is the failure path, and the failure path is the whole reason the wrapper felt convenient. A structured request can come back refused, can come back truncated because it hit the token cap mid-object, and can come back valid JSON that does not satisfy your schema on a provider that does not enforce it. Your replacement needs an explicit branch for each, and the truncation one in particular is only detectable by checking the finish reason before you attempt to parse.

The other thing to decide is what happens on the second provider. The native schema-enforcement mechanisms are not the same shape across vendors — one attaches a schema to the response format, another expresses it as a tool the model is required to call — which is precisely the abstraction the wrapper was giving you. The general treatment of structured output support covers the mechanisms; the migration point is that you are taking ownership of choosing between them.

Usage and streaming

Token usage is easier direct than through the wrapper, because it is just a field on the response — read usage and log it. The catch is streaming: on a streamed call the usage totals are not on the chunks by default. The OpenAI shape requires you to opt in with a stream option requesting usage, and the totals then arrive on a final chunk that carries no choices at all. Code that reads chunk.choices[0] unconditionally crashes on precisely the chunk you asked for.

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},
)

text, usage = [], None
for chunk in stream:
    if chunk.usage is not None:
        usage = chunk.usage          # final chunk: choices is empty
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta
    if delta.content:
        text.append(delta.content)

That if not chunk.choices: continue line is the single most important defensive line in a streaming client, and the wrapper was writing it for you. It is also load-bearing for reasons beyond usage — the streaming parser page covers the other chunks that arrive with no choices.

Also note what you lose in the accumulation: the wrapper handed you a finished message object at the end of a stream, with tool calls reassembled. Doing it yourself means accumulating text deltas and, if you use tools, reassembling fragmented tool-call arguments by index. That is a real piece of code and it belongs in your one internal function, written once.

Doing it incrementally

1. Write one internal function with the signature your application wants — messages in, text or a parsed object out — and implement it with the wrapper. Change nothing else. This is the whole migration; the rest is moving code behind it.
2. Move call sites onto that function one at a time, and keep going until nothing outside it imports LangChain.
3. Reimplement the function body with the provider SDK, behind a flag or an environment variable, so both implementations exist at once.
4. Run your regression suite against both. Compare outputs on a fixed seed and temperature zero where you can; where you cannot, compare the structured fields rather than the prose.
5. Add the usage logging, the finish-reason check and the streaming accumulation to the new implementation before you cut over, not after. These are the three things the wrapper was doing that you will not notice missing until an incident.
6. Cut over, run for a fortnight with both implementations still importable, then delete the old body and drop the dependency.

The reason to keep both alive for a fortnight is not caution about the rewrite. It is that the wrapper was normalising provider quirks you have never seen, and a fortnight is roughly how long it takes to meet the rarer ones — the refusal, the truncated object, the empty completion — in production traffic.