When a Framework Earns Its Place

A framework is not one decision. It is four, bundled: provider abstraction, control flow, integrations and operations. Most teams need one or two of those and adopt all four, which is why the regret arrives around month six rather than week one. Name which of the four you need and the question stops being religious.

The four jobs a framework does

Every AI framework — LangChain, LlamaIndex, Haystack, Semantic Kernel, the Vercel AI SDK, CrewAI — is some mixture of the same four jobs. The mixture differs, the marketing differs, the jobs do not.

• Provider abstraction. One call shape across OpenAI, Anthropic, Google, Mistral and whatever you run locally. The pitch is that changing model is a configuration change. It largely is, for the fields every provider has.
• Control flow. Chains, graphs, agent loops, branching, parallel fan-out, retries and — the genuinely hard one — state that survives a process restart. This is the job with the most real engineering in it.
• Integrations. Document loaders, text splitters, vector store adapters, tool wrappers, memory backends. The value is breadth on day one: a hundred connectors you did not write.
• Operations. Tracing, token accounting, callbacks, evaluation hooks, run replay. Frequently the reason a team that wanted to leave stays.

The important property is that these four are separable. Almost nothing forces you to buy them together, and the most common expensive mistake is adopting a whole framework for job three — an integration you would have written in forty lines — and paying for jobs one, two and four in ways that only become visible once the application is in production.

What the loop looks like without one

Before pricing the alternative it is worth being precise about what you are avoiding writing. The core of an agent — the thing frameworks are usually adopted for — is a loop that calls a model, notices it asked for a tool, runs the tool, appends the result and calls again. It is about forty lines.

# The whole of it. Fields shown are the OpenAI-compatible chat shape,
# which every major provider and every local server now speaks.

def run(messages, tools, tool_impls, max_turns=8):
    for turn in range(max_turns):
        reply = client.chat.completions.create(
            model=MODEL,
            messages=messages,
            tools=tools,
        ).choices[0].message

        messages.append(reply)

        calls = reply.tool_calls or []
        if not calls:
            return reply.content            # the model is done

        for call in calls:
            name = call.function.name
            args = json.loads(call.function.arguments)
            try:
                result = tool_impls[name](**args)
            except Exception as exc:        # a failed tool is a message,
                result = {"error": str(exc)}  # not an exception in your loop
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result),
            })

    raise RuntimeError("agent did not finish in %d turns" % max_turns)

That is not a strawman version. It is the shape of the agent loop as every framework implements it, minus the abstractions. If your application is one of these, running inside a request handler, finishing in seconds, with four tools you wrote yourself, then the forty lines are the correct amount of code and a framework is a net loss.

What a framework adds beyond it is real, though, and worth naming precisely: durable state so a run can resume mid-way after a crash or a human approval; structured concurrency so two branches run at once without you writing the join; streaming that survives being nested three calls deep; and a trace that shows which of the eleven model calls in a run produced the wrong answer. Every one of those is genuinely annoying to build and genuinely annoying to get right.

What each job costs you

Lock-in is not a slogan. It is a specific set of things you will have to rewrite if you leave, and it differs sharply by job.

The pattern in that table is the whole recommendation: pay for control flow if your control flow is hard, be sceptical about the other three, and keep operations on an open format so it never becomes the reason you stay.

A decision procedure that takes an afternoon

1. Draw your control flow. Not the happy path — the real one. Does it contain a cycle? A branch chosen by the model? A step where a human approves something hours later? A fan-out that joins? If the answer to all four is no, you have a pipeline, not an agent, and you do not need an orchestration framework.
2. Ask whether a run must survive a restart. This is the single highest-value question on the list. If a deploy in the middle of a twenty-minute run must not lose it, you need durable checkpointed state, and that is worth adopting a framework for. If every run finishes inside one HTTP request, it is not.
3. Count the integrations you need on day one. Not the number in the README. The actual list. Teams routinely adopt a framework for its hundred loaders and use two, both of which are a library call and a for-loop.
4. Name who reads the traces. If the answer is “nobody yet”, operations is not a reason. If it is a support engineer at 2am, it is the strongest reason on this list — though see the last row of the table above.
5. Build the hardest step twice, in one day. Not the demo. The step you are least sure about. A day of this tells you more than a week of comparison reading, because the thing you are testing is not whether the framework can do it but whether you can work out why it did what it did.
6. Decide per job, not per framework. “We use LangGraph for orchestration and call provider SDKs directly inside the nodes” is a coherent, common and cheap position. “We use LangChain” is not a decision, it is a default.

Adopt at the seam, not at the centre

If you do adopt one, the difference between a framework you can leave and a framework you cannot is entirely about where you let it touch. Four rules cover most of it.

• Your prompts live in your files. Plain text or templates you own, versioned in your repository, not embedded in framework classes. Prompts are the highest-churn, highest-value asset in the application and they should not need an import to read.
• Your domain types are yours. Pydantic or Zod models defined in your own module, passed into the framework, not framework document objects leaking into your business logic. This one rule does more to bound a migration than everything else combined.
• Framework code stays in one directory. Ifimport langchain appears in forty files, you have adopted it at the centre. If it appears in three, you have adopted it at a seam.
• Provider calls go through one function you wrote. Even if that function currently delegates straight to the framework. It is the seam that makes fallback chains and per-request cost accounting possible later without touching the application.

When the answer changes

The decision is not permanent in either direction, and there are four fairly reliable triggers for revisiting it. Durability becoming a requirement — the first time a deploy kills a long run in production — is the usual reason to adopt one. Team size is the second: a loop one person understands is fine, a loop eight people extend needs a vocabulary, and a framework supplies one.

In the other direction: when you find yourself reading the framework’s source to work out what prompt it sent, the abstraction has stopped paying. And when the release cadence outruns your ability to test upgrades — when you are pinned three minor versions back because the last bump broke something you could not diagnose — you are maintaining the framework rather than the application. That is the point at which a staged removal becomes cheaper than staying.