LangChain: The Parts Worth Using

LangChain stopped being one library some time ago. It is now a core package of interfaces, a large set of separately versioned integration packages, and a sibling project for orchestration. Knowing which of those you are importing is most of what separates a maintainable LangChain application from the ones people write blog posts about abandoning.

What LangChain actually is now

The split matters more than any individual class, because it is what decides how much of the library you inherit.

The one abstraction: the runnable

If you learn one thing about LangChain, learn this. Almost every component implements the same interface: it has invoke for one input, batch for many, and stream for incremental output, with async variants of each. Anything implementing that interface can be composed with anything else implementing it using the pipe operator.

# The composition model, which is the actual content of the library.
chain = prompt | model | parser

chain.invoke({"question": "..."})        # one
chain.batch([{...}, {...}, {...}])       # many, concurrently
for chunk in chain.stream({...}): ...    # incremental

That is genuinely useful, and it is the part worth keeping. Composing three steps gets you batching and streaming across all three for free, including the awkward case where streaming has to pass through a parser. A plain function does not give you that.

It is also the source of the library’s worst ergonomics. When a chain of six runnables raises, the traceback is about the runnable machinery, not about your code, and the input to step four is whatever step three decided to emit. Keep chains short — three or four components — and the debugging cost stays manageable.

Build it the framework way

The application: answer questions over a folder of markdown files with citations. The framework version reads roughly like this, in the idiom the documentation encourages.

# Sketch, not a paste-and-run script: class names and import paths
# differ by version. The structure is the point.

loader     = DirectoryLoader("./docs", glob="**/*.md")
splitter   = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
embeddings = <provider embeddings class>()
store      = <vector store class>.from_documents(splitter.split_documents(loader.load()),
                                                 embeddings)
retriever  = store.as_retriever(search_kwargs={"k": 5})

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer only from the context. Cite the source of each claim."),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),
])

chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)

chain.invoke("How do I rotate an API key?")

About twenty lines, and it works. It is also the version that produces the complaint everyone eventually makes: when the answer is wrong, you cannot see the prompt. The retrieved chunks were joined by a helper, the template filled them in, and what actually reached the model is three abstractions away from anything you wrote.

Build it again, smaller

Now the same application, with the framework used only where it is adding something. Chunking is a function. Retrieval is a query against whatever store you were going to use anyway. The prompt is a string you can print.

def chunks(text, size=1000, overlap=150):
    step = size - overlap
    return [text[i:i + size] for i in range(0, len(text), step)]

def build_prompt(question, docs):
    context = "\n\n".join(
        f"[{i + 1}] source: {d['path']}\n{d['text']}" for i, d in enumerate(docs)
    )
    return (
        "Answer only from the context. Cite each claim as [n].\n\n"
        f"Context:\n{context}\n\nQuestion: {question}"
    )

def answer(question):
    docs = store.search(embed(question), k=5)   # your store, your schema
    prompt = build_prompt(question, docs)
    log.debug("prompt=%s", prompt)              # the line the framework cost you
    return client.chat(prompt)

Roughly the same length, and every intermediate value is a Python object you can print. The naive splitter above is deliberately naive — real chunking wants structure-aware boundaries — but the point stands: the splitter was never the hard part, and wrapping it in a class did not make it better.

What earned its place

Building it both ways is not an argument for zero framework. Four things in the framework version are genuinely worth importing, and they are not the four most tutorials emphasise.

• The message and chat-template types. Getting system, human, assistant and tool messages right across providers, including multi-part content, is fiddly and boring. This is a real saving.
• batch with bounded concurrency. Running two hundred prompts with a concurrency limit, retries and ordered results is code you would otherwise write badly at least once.
• Streaming through a parser. Emitting partial structured output as it arrives is the awkward case, and the runnable protocol handles it without you writing an incremental parser. Related: streaming JSON parsing.
• The callback and tracing hooks. One line of setup gets every model call, its inputs and its token counts into a trace. Whether that is worth the dependency depends on whether you have already got observability from somewhere else.

Everything else in the first version — the loader, the splitter, the retriever wrapper, the passthrough plumbing — was a for-loop with a class around it.

Where it hurts

Three failure modes account for most of the pain, and all three are predictable enough to plan around.

You cannot see the prompt

The most common and most damaging. Fix it on day one: log the fully rendered prompt for every call, in development at least. If the framework makes that hard for a given component, that component is one you should be writing yourself.

Dependency surface

Integration packages pull transitive dependencies you did not choose, and version conflicts between two integration packages are a routine way to lose an afternoon. Install the provider packages you need rather than an umbrella package, and pin them.

Upgrades are not routine

Class moves and renames between majors are normal here. Budget for it: keep framework imports in a small number of files, keep your prompts and your domain types outside the framework, and an upgrade becomes a morning rather than a sprint. That is the same discipline that makes leaving cheap if you decide to.

A checklist before it ships

The gap between a LangChain prototype and a LangChain service is six items, none of which the quickstart mentions and all of which are an afternoon in total.

1. Pin every package, including the integration packages. They version independently, so an unpinned integration can move under a pinned core. A lockfile is the requirement; a range in the manifest is not.
2. Log the rendered prompt and the raw response. At debug level in production, at info level in staging. Without it, an incident becomes an exercise in reading library source under time pressure.
3. Set a timeout and a retry policy explicitly. Framework defaults for both exist and are unlikely to match what your request handler can tolerate. A model call with no deadline is a request that hangs until something else gives up.
4. Bound concurrency on every batch call. The convenience of running two hundred inputs at once is also the fastest route to a provider rate limit, and the failure arrives as a burst rather than as a trickle.
5. Record token usage yourself. Usage is on the response; get it into your own metrics rather than relying on a callback you may remove later. Cost per request is the number everyone asks for in month two — see cost per request.
6. Write down which imports you are relying on. A short list in the repository of the framework surface you actually use is what makes the next upgrade a scoped task instead of an exploration.

The last one has an unexpected side effect: teams that write the list usually find it has four entries, which is the moment the question in whether a framework earns its place stops being theoretical.