Migrating From a Legacy Completions Endpoint

Something that had run untouched for two years stopped. The error is either an SDK complaining about an attribute, or a 404 about a model name, or a field on the response that is suddenly undefined. All three have the same underlying cause and the same rewrite.

The errors you are seeing

There are four ways this arrives, and which one you get tells you where in the stack the change happened.

An attribute that no longer exists

AttributeError: module 'openai' has no attribute 'Completion'
AttributeError: module 'openai' has no attribute 'ChatCompletion'

This is not the endpoint being removed. It is the Python SDK’s 1.0 rewrite, which replaced module-level helpers with methods on a client object, and it fires the moment somebody bumps the dependency. Recent versions raise a more explicit error than a bare AttributeError, naming the version boundary and pointing at the migration. Either way, the code you have was written against the pre-1.0 interface, and no amount of changing the model name will help until the client is constructed properly.

A model that is gone

{"error": {"message": "The model 'text-davinci-003' does not exist or you do not have access to it",
           "type": "invalid_request_error", "code": "model_not_found"}}

A 404 with a model_not_found code. The endpoint is still there; the model behind it has been retired. This is the one that arrives without any change on your side, on the sunset date, which is why tracking deprecation dates is worth doing before it happens rather than after. The model-not-found error has its own page for the cases where the name is right and the access is not.

A response field that is undefined

TypeError: Cannot read properties of undefined (reading 'text')
KeyError: 'text'

Somebody changed the endpoint but not the response handling. A completions response puts the generated text at choices[0].text; a chat completions response puts it at choices[0].message.content. The request succeeded; the read failed. This is the failure that reaches production most often, because the request change passes a smoke test that only asserts a 200.

A deprecation notice with no error at all

Before any of the above, you will usually have received a written notice — in a dashboard banner, a changelog entry or an email — with wording of the form “this model is deprecated and will be shut down on” a date, generally with a named replacement. Deprecated and shut down are two different states, sometimes months apart: the first is a warning, the second is the 404 above. Treat the notice as the deadline and not the shutdown, because after the shutdown your options are limited to the rewrite done in a hurry.

The call-site rewrite

The endpoint path changes from the completions route to the chat completions route, the SDK method changes with it, and the single prompt string becomes a message array. In Python, before and after:

# before — pre-1.0 SDK, completions endpoint
import openai
openai.api_key = KEY

resp = openai.Completion.create(
    model="text-davinci-003",
    prompt=prompt_text,
    max_tokens=256,
    temperature=0,
    stop=["\n\n"],
)
answer = resp["choices"][0]["text"].strip()

# after — 1.x SDK, chat completions endpoint
from openai import OpenAI
client = OpenAI()                      # reads OPENAI_API_KEY from the env

resp = client.chat.completions.create(
    model="<a current chat model id>",
    messages=[
        {"role": "system", "content": standing_instruction},
        {"role": "user",   "content": variable_content},
    ],
    max_tokens=256,                    # see the note below on newer models
    temperature=0,
)
answer = resp.choices[0].message.content.strip()

Three things in that diff are easy to miss. The client is constructed rather than configured globally, so anything that set openai.api_key, openai.api_base or a proxy at module level has to move into the constructor. The response is an object with attributes rather than a dictionary, so subscript access on the result fails even where the field name is unchanged. And the stop parameter has been dropped — a double-newline stop that existed to end a completion is one of the four things the prompt rewrite deletes, and leaving it in silently truncates any answer containing a blank line.

Reading the response

The response envelope is similar enough to look interchangeable and is not. Both shapes have a choices array with an index and a finish_reason, and both have a usage object with prompt_tokens, completion_tokens and total_tokens. What differs:

• The text moves. choices[0].text becomes choices[0].message.content, and the message also carries a role. On a turn where the model called a tool, the content can be null and the substance is in tool_calls instead — a case the old endpoint had no equivalent for and which will crash a handler that assumes a string.
• Finish reasons gain values. The old set was essentially stop and length. The chat endpoint adds tool_calls and content_filter, and a legacy handler that branches only on length treats all of them as normal completion. Every finish_reason value enumerates them.
• Log probabilities are shaped differently. The completions endpoint took an integer logprobs parameter; the chat endpoint takes a boolean logprobs and a separate top_logprobs integer, and returns them in a different structure under the choice. Any code reading the old structure needs rewriting, not renaming — see the top_logprobs parameter.
• Streaming chunks differ. Completion chunks carried text; chat chunks carry a delta object whose content may be absent on the first and last chunks. A parser that concatenates a field unconditionally emits undefined into the output string. The streaming chunk format covers the shape.

Parameters with no counterpart

Most parameters carry over unchanged. Four do not, and each needs a decision.

• Suffix. Text after the insertion point, for fill-in-the-middle. No chat equivalent. If your feature depends on it, it needs an infilling-capable model rather than a parameter.
• Echo. Returning the prompt with the completion. No equivalent, and its main use — scoring existing text via log probabilities — is not a thing the chat endpoint does.
• Best-of. Generating several candidates server-side and returning the best by log probability. Where it is absent, the client-side approximation is to request several completions and rank them yourself, which costs the same tokens but requires you to define “best”.
• Bare prompt strings anywhere else. Audit for other code paths that build a prompt string — batch jobs, evaluation harnesses, notebooks. They will not error until they run, and an evaluation harness still calling the old endpoint produces numbers you will compare against new ones without noticing.

Doing it in a safe order

The rewrite touches the request, the response handling and the prompt at once, which makes a regression hard to attribute. Separate them.

1. Capture a set of real inputs and their current outputs from the old endpoint, while it still works. This is your only baseline and it expires on the shutdown date.
2. Upgrade the SDK and fix the construction errors, with no other change. The code will not run against the old endpoint if the endpoint or model is already gone, which is why the baseline comes first.
3. Change the endpoint and wrap the old prompt string in a single user message, changing nothing about the text. Fix the response paths. The output will be worse than the baseline — that is expected and is not the endpoint’s fault.
4. Now rewrite the prompt properly: instruction to the system position, few-shot pairs to real turns, trailing cue and separators deleted.
5. Diff against the baseline on the captured inputs. Anything that reads the model’s output with a regex or an exact match is where the breakage will be, because output length and formatting are the things that change most.
6. Search the repository for the old method name, the old model string and .text reads on a response object before declaring it done.