generateContent and streamGenerateContent: Gemini's Two Response Modes

Gemini exposes streaming as a separate method rather than as a flag on the request body. streamGenerateContent returns the same response type as generateContent, repeatedly—and a query parameter you may not have set decides whether those repeats arrive as server-sent events or as a JSON array delivered in pieces.

Two methods, one response type

There is no "stream": true in a Gemini request body. The choice is in the URL:

POST .../v1beta/models/gemini-2.5-flash:generateContent
POST .../v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse

The request body is identical for both. What differs is that generateContent returns one GenerateContentResponse when generation is complete, and streamGenerateContent returns a sequence of them as generation proceeds. Google’s generateContent reference documents both against the same response schema, which is the fact that makes the streamed shape easy to read: a chunk is not a special “delta” type with its own fields. It is a whole response object that happens to contain only the text produced since the last one.

The non-streamed response, for reference:

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [{ "text": "The three regions are London, Manchester and Leeds." }]
      },
      "finishReason": "STOP",
      "index": 0
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 24,
    "candidatesTokenCount": 11,
    "totalTokenCount": 35
  },
  "modelVersion": "gemini-2.5-flash"
}

alt=sse, and what happens without it

alt=sse is the framing selector, and omitting it is the single most common reason a Gemini stream “does not stream”.

With ?alt=sse, the response is text/event-stream and each chunk is one SSE event: the literal bytes data: followed by one JSON object, followed by a blank line. This is the form every SSE client and every Google SDK expects.

data: {"candidates":[{"content":{"role":"model","parts":[{"text":"The three "}]}}]}

data: {"candidates":[{"content":{"role":"model","parts":[{"text":"regions are London, "}]}}]}

data: {"candidates":[{"content":{"role":"model","parts":[{"text":"Manchester and Leeds."}]},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":24,"candidatesTokenCount":11,"totalTokenCount":35},"modelVersion":"gemini-2.5-flash"}

Without it, the response content type is application/json and the body is a single JSON array of response objects, written out incrementally: an opening [, then each object separated by commas as it is produced, then a closing ]. It is still streamed over the wire, but it is not parseable until the last byte arrives unless you have an incremental JSON parser, so a naive client blocks until the end and looks exactly like a non-streaming call that is slower than it should be.

What is in a chunk

Each chunk is a GenerateContentResponse, and the fields behave differently from one another:

• candidates[0].content.parts — incremental. The text in a chunk is the new text, not the running total. Concatenate; do not replace.
• candidates[0].finishReason — absent on every chunk except the last, where it carries the same values as in a non-streamed response.
• usageMetadata — the token counts. Read them from the final chunk; that is where the complete candidatesTokenCount and totalTokenCount are.
• modelVersion — which concrete model served the request, useful when you called an alias rather than a pinned version.
• promptFeedback — present only if the prompt itself was blocked, in which case there may be no candidate text at all.

Chunk boundaries are not token boundaries and not sentence boundaries. A chunk can carry one token or a whole paragraph, and it can split a multi-byte character sequence or a markdown fence across two chunks. Never run a regex over a single chunk; run it over the accumulated string.

Function calls behave differently from text. A functionCall part arrives as a complete object—name and decoded args together—rather than as incremental fragments of an argument string that you have to reassemble and parse. If you are porting a tool loop from an API that streams argument deltas, that reassembly code has nothing to do here and its buffering logic will produce an empty result.

Accumulating correctly

The Google Gen AI SDKs handle framing for you and yield the same chunk objects:

from google import genai

client = genai.Client()

stream = client.models.generate_content_stream(
    model="gemini-2.5-flash",
    contents="List our three UK regions.",
)

pieces = []
usage = None
finish = None

for chunk in stream:
    if chunk.text:
        pieces.append(chunk.text)
        print(chunk.text, end="", flush=True)
    if chunk.usage_metadata:
        usage = chunk.usage_metadata
    for candidate in chunk.candidates or []:
        if candidate.finish_reason:
            finish = candidate.finish_reason

answer = "".join(pieces)

The guard on chunk.text is not defensive padding. A chunk can legitimately carry no text—a chunk whose only content is a function call, or the final chunk that carries only usage metadata and a finish reason—and the SDK returns None there rather than an empty string.

Cancelling, and the proxies in between

Streaming makes the HTTP connection part of your application’s behaviour, which is where most of the surprises that are not about JSON shapes come from.

Cancelling. There is no cancel endpoint. You stop a generation by closing the response body, and that is a real cancellation rather than a client-side discard—generation stops when the connection goes. What you are billed for is what was produced up to that point, which you can no longer read from usageMetadata because the final chunk never arrived. If accurate per-request cost accounting matters, count the tokens you received yourself, or accept that cancelled requests are under-counted in your own records.

Buffering intermediaries. A reverse proxy that buffers responses will hold every chunk and deliver them together at the end, which turns a stream back into a slow non-streaming call without producing any error. If your stream works locally and does not work behind your infrastructure, this is the first thing to check: nginx needs proxy_buffering off for the route, and many CDN and edge-function layers have an equivalent setting or an incompatibility they document.

Re-streaming to your own client. Passing chunks straight through to a browser is the usual design, and it has a consequence worth deciding deliberately rather than discovering: you cannot un-send text. If a generation is blocked partway, or produces something you would have filtered, it is already rendered. The alternative is to buffer server-side, check the finish reason, and then release—which costs you the entire perceived latency benefit that made streaming worth doing. Most products land on streaming with a visible retraction path rather than one extreme or the other, and it is better to choose that than to inherit it.

Idle timeouts. A thinking model can produce no output tokens for many seconds while it reasons, and an intermediary with a short idle timeout will cut a connection that is working perfectly well. Time-to-first-token on a reasoning model is not comparable to a non-reasoning one, and gateway timeouts sized for the latter are a common cause of streams that die at a consistent number of seconds.

How a stream ends, including badly

Three endings are worth handling separately, because two of them deliver a partial answer that looks complete.

1. Clean. Final chunk carries finishReason: "STOP" and full usageMetadata. The accumulated string is the answer.
2. Truncated. Final chunk carries finishReason: "MAX_TOKENS". You already streamed most of an answer to the user and it stops mid-sentence. This is why the finish reason must be checked after the loop even though the text has already been displayed—see the full list of finishReason values.
3. Blocked mid-generation. Final chunk carries finishReason: "SAFETY" or another block value. Everything streamed before that point is still on the user’s screen. If your product cannot show a partially generated answer that was then blocked, you cannot stream directly to the user—you have to buffer and check.

A dropped connection is a fourth case and it is not distinguishable from a clean end by content alone: the body simply stops. The only reliable signal is that no chunk carried a finishReason. Treat “stream ended with no finish reason” as an error rather than as success, or you will silently serve half-answers whenever a proxy times out.