Migrating Away From a Provider's Native Assistants API

The Assistants API stores your conversation, decides what fits in the context window, runs the tool loop and manages a vector index. Leaving it means writing four things, and only the first is easy.

What the managed API was doing

The object model is small and each object is standing in for code you are about to write. An assistant holds a model, instructions and a tool list. A thread holds an ordered list of messages, server-side, indefinitely. A run executes one assistant against one thread and moves through a status machine — queued, in_progress, requires_action, completed, failed, cancelled, expired, incomplete. A run step records what happened inside it. And file_search (the tool formerly called retrieval) holds a managed vector store you never had to build.

Read that list as a bill of materials. The migration is four substitutions: a message store, a context assembler, a tool loop, and a retrieval pipeline. The first is a database table. The last is a project.

The state you now own

The thread is the obvious piece and the least interesting. Two tables — one for conversations, one for messages — reproduce it, with the caveat that you now have to decide things the managed API decided for you.

create table conversation (
  id           uuid primary key,
  user_id      uuid not null,
  system       text not null,           -- was the assistant's "instructions"
  model        text not null,           -- pin a dated snapshot, not an alias
  created_at   timestamptz not null default now()
);

create table message (
  id             bigserial primary key,
  conversation_id uuid not null references conversation(id),
  seq            int  not null,          -- ordering you now own
  role           text not null,          -- system | user | assistant | tool
  content        jsonb,                  -- text, or content blocks
  tool_calls     jsonb,                  -- assistant turn: the calls it asked for
  tool_call_id   text,                   -- tool turn: which call this answers
  input_tokens   int,
  output_tokens  int,
  created_at     timestamptz not null default now(),
  unique (conversation_id, seq)
);

Three columns there are load-bearing and are the ones usually left out on a first attempt. seq exists because ordering by timestamp breaks the moment two messages land in the same millisecond, and a reordered tool result is an invalid conversation rather than a slightly odd one. tool_call_id exists because a tool result message is only valid if it references the identifier of the call it answers — omit it and the API rejects the turn. And the two token columns exist because you are about to need a running total, which the managed API kept for you.

The harder half is the context assembler. A thread can grow indefinitely; a context window cannot. The managed API resolved that with a truncation strategy — configurable per run, defaulting to dropping older messages to fit — and now you resolve it. The rule is not “keep the last N messages”: dropping a message that contains a tool call while keeping its result, or vice versa, produces a request the API refuses. Truncate in whole turns, always keep the system message, and never split a call from its result.

def assemble(system, turns, budget_tokens, count):
    """turns: list of turn groups, oldest first. A turn group is
       [assistant-with-tool_calls, tool-result, tool-result, ...] or [single message].
       Truncating anything smaller than a group produces an invalid request."""
    kept, used = [], count(system)
    for group in reversed(turns):                 # newest first, drop from the front
        cost = sum(count(m) for m in group)
        if used + cost > budget_tokens:
            break
        kept.append(group)
        used += cost
    msgs = [{"role": "system", "content": system}]
    for group in reversed(kept):
        msgs.extend(group)
    return msgs, used

Reserve headroom in budget_tokens for the output cap, because input plus output share the window — the arithmetic is on context window versus max tokens, and the edge case is worth a test of its own (behaviour at the context window edge).

The run loop, explicitly

A run is a loop, and the status machine you were polling collapses into ordinary control flow. The one status that carried real semantics is requires_action: the model asked for tools, the run paused, and you had a bounded window to submit outputs before it expired. In your own loop that pause is just the top of the next iteration, and the expiry is whatever timeout you choose.

MAX_STEPS = 8          # was the run's implicit ceiling; now an explicit one

def run(conversation_id, user_text):
    append(conversation_id, role="user", content=user_text)
    for step in range(MAX_STEPS):
        msgs, _ = assemble(*load(conversation_id))
        r = client.chat.completions.create(
            model=MODEL, messages=msgs, tools=TOOLS,
            tool_choice="auto", max_completion_tokens=1024,
        )
        choice = r.choices[0]
        append(conversation_id, role="assistant",
               content=choice.message.content,
               tool_calls=choice.message.tool_calls,
               usage=r.usage)

        if choice.finish_reason != "tool_calls":
            return choice.message.content          # stop | length | content_filter

        for call in choice.message.tool_calls:     # was submit_tool_outputs
            result = dispatch(call.function.name,
                              json.loads(call.function.arguments))
            append(conversation_id, role="tool",
                   tool_call_id=call.id, content=json.dumps(result))
    raise StepBudgetExceeded(conversation_id)

Four things in that loop are decisions the managed API made silently and now belong to you. MAX_STEPS is a termination guarantee — without it a model that keeps requesting tools loops until your budget is gone (a guard worth testing). Dispatch must be idempotent or retried tool calls will double side effects (retrying a tool call without duplicating side effects). A finish_reason of length mid-loop means a truncated tool-call payload and must be handled, not treated as an answer. And the tool result must be appended even when the tool failed — a conversation containing an assistant turn with tool calls and no matching results is invalid on the next request, so an exception in dispatch has to become a tool message describing the failure.

Streaming is the other thing you rebuild here. The managed API streamed run events; a chat completion streams token deltas and tool call arguments arrive fragmented across chunks, so the assembly of a complete tool call from a stream is code you now write. The shapes are covered on parsing streamed JSON and handling partial JSON mid-stream.

Retrieval, which does not port

Everything above is mechanical. This part is not, and it is where the migration estimate goes wrong. The managed file-search tool took uploaded files, chunked them, embedded them, stored the vectors, retrieved on each run and injected the results — and it exposed almost none of those decisions, which means you cannot replicate its behaviour by reading a configuration. You are building a retrieval pipeline, not porting one.

• Extraction and chunking — the chunk size and overlap that worked inside the managed tool are not published, so this is tuned from scratch against your documents. Start at RAG chunking.
• An embedding model and a vector store — plus a reindex path, because changing the embedding model invalidates every stored vector.
• A retrieval step in the loop — the managed version retrieved implicitly. Yours is either an explicit search before the call or a tool the model chooses to invoke, and the two behave differently enough that it is a design decision rather than a detail.
• Citations — the managed API returned annotations pointing at source files. Reproducing that means carrying chunk identifiers through retrieval into the prompt and back out of the answer (RAG citations).

If your assistant used file search at all, budget for this as the majority of the work. If it did not, the migration is genuinely a week-shaped task rather than a quarter-shaped one, and knowing which of those you are in is the first question to answer.

Cutting over without losing threads

1. Export the threads first, before anything else. Page every thread and every message out through the API and store the raw JSON. Server-side conversation state is the one thing in this migration you cannot rebuild after the fact, and the export is cheap while the API is live.
2. Import into your own tables, preserving order and keeping the original thread and message identifiers in a column. You will need them to reconcile.
3. Run your loop in shadow mode on a copy of live traffic, writing to your own store and discarding the output. Compare structure, not text — the harness on testing prompts across versions is the right instrument, because the underlying model is the same and the differences you are hunting come from your context assembly.
4. Cut over new conversations only. Existing threads continue on the managed API until they go idle. This avoids the worst class of bug — a conversation whose first half was assembled by one context policy and second half by another.
5. Re-export before you delete anything. Threads that were active during the cutover accumulated messages after your first export.
6. Then stop dual-running. Pick the date in advance; dual-running has a way of becoming permanent, and its cost is covered on what you lose leaving a managed threads API.