Testing How Your App Handles an Empty Completion From the Model

TypeError: Cannot read properties of null (reading 'trim') in JavaScript, or AttributeError: 'NoneType' object has no attribute 'strip' in Python. The stack points at your formatting code. The cause is two frames earlier, in a response you assumed always had text in it.

The error you pasted into search

Three error strings account for nearly all of these, and each one tells you something different about which shape you got.

• Cannot read properties of null or 'NoneType' object has no attribute — the message object exists and its content field is null. This is the common one, and the usual cause is a tool call: the model returned a function invocation instead of text, and the text field is null by design.
• IndexError: list index out of range or Cannot read properties of undefined on choices[0] — the choices array is empty. The response arrived, was well-formed, and contains nothing.
• A validation error from your schema parser complaining that a required field is missing, when the model returned an empty string and JSON.parse was handed "". The reported error names your schema, which sends people to debug the wrong file.

None of these is a provider outage, and retrying blindly makes several of them worse. Which one you have determines whether a retry can help at all.

Six ways a response can be empty

Write one fixture per shape. They are static JSON, they cost nothing, and they are the entire test suite for this bug.

1. Content null, tool call present. The normal shape for a tool-calling turn. Code that reads the text before checking for tool calls crashes on the happy path of a feature it already supports.
2. Content an empty string, finish reason a normal stop. The model genuinely produced nothing. Common when the prompt ends in a way that makes an immediate stop token likely, or after a successful refusal that emitted no text.
3. Truncation at zero tokens. A length-based finish reason with no content, which happens when the output cap is very small or the prompt consumed the window. Retrying identically returns the same thing forever.
4. A content filter or refusal. The finish reason names a filter, or the message carries a separate refusal field with text in it while the content field is null. Reading only the content field loses the explanation you would want to show the user.
5. An empty choices array. Rare, and it defeats every null check written against the message object because the crash is on the index.
6. Whitespace only. A newline, a space, a zero-width character. Not null, not empty by a length check, and still empty for your purposes. This one gets past the first three fixes people write.

Providers differ in which of these they emit and what they name the reason field — one API family calls it a finish reason on the choice, another a stop reason on the message — so a codebase talking to more than one needs the normaliser below regardless of the crash.

One normaliser, tested against all of them

The wrong fix is a null check at the crash site. There will be four more crash sites, and each will get its own slightly different check. Put one function between the SDK and your code, give it a return type that makes the empty case impossible to ignore, and point every fixture at it.

// normalise.ts
export type Completion =
  | { kind: "text"; text: string }
  | { kind: "tool_calls"; calls: ToolCall[] }
  | { kind: "empty"; reason: EmptyReason; retryable: boolean };

export type EmptyReason =
  | "no_choices" | "null_content" | "whitespace_only"
  | "truncated" | "filtered" | "refusal";

export function normalise(res: unknown): Completion {
  const choice = (res as any)?.choices?.[0];
  if (!choice) return { kind: "empty", reason: "no_choices", retryable: true };

  const calls = choice.message?.tool_calls;
  if (Array.isArray(calls) && calls.length > 0) return { kind: "tool_calls", calls };

  if (choice.message?.refusal)
    return { kind: "empty", reason: "refusal", retryable: false };

  const raw = choice.message?.content;
  if (raw == null || String(raw).trim() === "") {
    const fr = choice.finish_reason;
    if (fr === "length") return { kind: "empty", reason: "truncated", retryable: false };
    if (fr === "content_filter") return { kind: "empty", reason: "filtered", retryable: false };
    return {
      kind: "empty",
      reason: raw == null ? "null_content" : "whitespace_only",
      retryable: true,
    };
  }
  return { kind: "text", text: String(raw) };
}

A discriminated union rather than a nullable string is doing real work here. Your call sites now cannot compile while ignoring the empty case, which is a stronger guarantee than any test, and the reason field turns a crash into a metric you can chart. The test is then one table:

it.each([
  ["tool call, null content", fixtures.toolCall, "tool_calls"],
  ["empty string", fixtures.emptyString, "empty"],
  ["truncated at zero", fixtures.truncated, "empty"],
  ["filtered", fixtures.filtered, "empty"],
  ["no choices", fixtures.noChoices, "empty"],
  ["whitespace only", fixtures.whitespace, "empty"],
  ["ordinary text", fixtures.text, "text"],
])("%s", (_n, res, kind) => expect(normalise(res).kind).toBe(kind));

it("does not treat a tool call as empty", () => {
  expect(normalise(fixtures.toolCall).kind).toBe("tool_calls");
});

Note the ordering inside the function: tool calls are checked before content, and the refusal field before the content emptiness test. Get those the other way round and the tool-calling path reports itself as empty, which is the second-most-common version of this bug and the one discussed in a tool call that appears not to fire.

Read the finish reason before you retry

The retryable flag is the point of the whole exercise. An empty result from a truncation or a content filter will be empty again on an identical request, and retrying it three times triples the cost of a guaranteed failure — the arithmetic in retry cost applies directly.

Assert the classification, not just the emptiness. A test that says the filtered fixture is not retryable and the null-content fixture is has captured the actual decision. Then assert what the caller does with it: a non-retryable empty produces a user-visible message and a counter increment; a retryable empty goes through your normal backoff and, if it is still empty, surfaces the same way rather than looping. Cap it at one or two attempts, because a model that produced nothing twice with the same prompt is telling you about the prompt.

Empty streams are a different bug

If you stream, none of the above fires, because there is no response object to inspect — there is a sequence of events that ends without ever carrying content. The three cases worth fixtures are a stream that opens and closes with no delta at all, one that emits only role and finish events, and one that emits a finish reason and then no terminator.

Test them by feeding a canned event sequence to your stream consumer rather than by opening a socket, and assert the consumer resolves with an empty classification instead of hanging. A stream that never finishes is the worst version of this: it holds a connection and a request slot, and it fails as a timeout somewhere unrelated. Give the consumer a deadline measured from the last received event, not from the start, and assert with fake timers that the deadline fires. Parsing structured output from a stream adds a further case: a stream that ends mid-object is empty as far as your parser is concerned, and the fixture for it is a truncated JSON prefix.