Skip to content
Multigrid

Streaming Output in Qwen: DashScope and OpenAI-Compatible Endpoints

12 min read · updated August 11, 2026

Alibaba serves the same Qwen model behind two HTTP interfaces with different streaming semantics. The difference that will actually bite you is not the JSON shape: it is that the native endpoint sends the whole answer so far in every event unless you ask it not to.

Two endpoints, one model

Model Studio exposes Qwen through a native DashScope API and through an OpenAI-compatible one, both documented in the Alibaba Cloud Model Studio documentation. The compatible endpoint lives under a /compatible-mode/v1 path and accepts the request body an OpenAI SDK already sends; the native one has its own body shape with input and parameters objects.

They are not two different services. The compatible endpoint is a translation layer over the same inference, which is the useful mental model: anything the native API can express that the OpenAI request schema has no field for is either unavailable through the compatible endpoint or smuggled through in a vendor-specific extra field.

The OpenAI-compatible shape

Set stream: true and you get server-sent events carrying chat.completion.chunk objects, terminated by a literal [DONE] sentinel that is not JSON:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1770000000,"model":"qwen-plus","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1770000000,"model":"qwen-plus","choices":[{"index":0,"delta":{"content":"Lis"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1770000000,"model":"qwen-plus","choices":[{"index":0,"delta":{"content":"bon"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1770000000,"model":"qwen-plus","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Deltas are incremental — you concatenate delta.content across chunks — and the first chunk carries the role with empty content. Usage is absent unless you ask for it with stream_options: {"include_usage": true}, which appends a final chunk carrying a usage object and an empty choices array. Client code that assumes every chunk has achoices[0] crashes on exactly that chunk, which is a fun one to debug because it only appears when you turn usage reporting on.

Reasoning models add a field the OpenAI schema does not have. Thinking-mode deltas arrive as delta.reasoning_content rather than delta.content, which is a genuine convenience — the endpoint has done the <think> parsing described in the thinking-mode page for you — but it means a strictly-typed OpenAI client will drop the field on the floor unless it tolerates unknown keys. If your reasoning output is mysteriously empty while the answer is fine, this is where to look first.

For the shape this is imitating, and the details it imitates closely enough to reuse a parser, see the OpenAI streaming chunk format.

The DashScope native shape

The native endpoint streams only when you send the header X-DashScope-SSE: enable. That is a header, not a body field, and forgetting it produces a perfectly valid non-streaming response that arrives all at once after a long silence — a symptom people usually misdiagnose as latency.

The events carry more of the SSE envelope than the compatible endpoint does, and the payload is nested under output rather than choices:

id:1
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"role":"assistant","content":"Lis"},"finish_reason":"null"}]},"usage":{"input_tokens":24,"output_tokens":1},"request_id":"..."}

id:2
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"role":"assistant","content":"bon"},"finish_reason":"stop"}]},"usage":{"input_tokens":24,"output_tokens":2},"request_id":"..."}

Three differences beyond the nesting are worth noting. There is no [DONE] sentinel — the stream ends when the connection ends, and you detect completion from finish_reason. Usage is present on every event and is cumulative, so you can display a running token count without waiting. And finish_reason is the string "null" rather than a JSON null while generation is in progress, which is a real quirk and one that a truthiness check in a dynamically-typed language will get exactly backwards.

The cumulative-output default

This is the single most important difference and the reason this page exists. On the native endpoint, streamed events are cumulative by default: each event contains the entire text generated so far, not the new fragment. The parameter that changes it is incremental_output, and it defaults to false.

{
  "model": "qwen-plus",
  "input": {
    "messages": [{"role": "user", "content": "What is the capital of Portugal?"}]
  },
  "parameters": {
    "incremental_output": true,
    "result_format": "message"
  }
}

Concatenating cumulative events the way you would concatenate deltas produces the classic output: LisLisbLisboLisbon. It is instantly recognisable once you have seen it, and completely baffling the first time, because the model is behaving perfectly and the bug is one boolean deep in a parameters object.

There is a bandwidth argument too. A cumulative stream re-sends the whole answer on every token, so the bytes over the wire grow with the square of the response length. For a 2,000-token answer that is a material amount of redundant traffic. Set incremental_output: true unless you have a specific reason to want whole-state events — the one legitimate case being a UI that replaces rather than appends, where cumulative events are naturally idempotent and a dropped event costs you nothing.

Set result_format: "message" as well. The older "text" format returns output.text instead of output.choices, and code written against one shape does not read the other.

What breaks in a ported client

The compatible endpoint is compatible enough that a client written against another provider usually connects on the first attempt, which is precisely why the remaining differences are expensive: they surface in production rather than during integration. These are the ones worth checking before you ship.

  • The base URL is regional. Model Studio serves mainland-China and international endpoints on different hosts, and an API key issued in one region does not authenticate against the other. The failure is a 401 on a key you can see is valid, which sends people looking at their credentials rather than their host.
  • Model names are not the open-weight names. The hosted catalogue uses commercial identifiers — the qwen-plus and qwen-turbo family — and their capacities are configured independently of the Hugging Face checkpoint that shares a generation with them. Sending a repository name as model gets you a model-not-found error; assuming the hosted model has the open checkpoint’s context window gets you something worse, which is a limit you discover under load.
  • The usage chunk has no choices. Covered above, and it deserves repeating here because it is the single most common crash when porting a working client: the final chunk under include_usage carries an empty array, and chunk.choices[0].delta throws on it.
  • Reasoning arrives in a field your types do not have. A strictly-decoded client — anything in a statically-typed language that rejects unknown keys rather than ignoring them — either drops reasoning_content silently or fails to parse the chunk entirely, depending on how strict it is.
  • Parameters without an OpenAI field are silently dropped. The compatible surface can only carry what the OpenAI request schema can express. A vendor-specific control you send as a top-level key is not an error; it is ignored, and the request succeeds with the default behaviour you were trying to change.

The general shape of the risk is that a compatibility layer converts hard failures into soft ones. An incompatible API tells you immediately; a compatible-except-here API tells you when a user reports that the reasoning panel is empty.

Which to build against

Build against the OpenAI-compatible endpoint unless you need something it cannot express. The reasons are not about elegance: your SDK, your retry logic, your streaming parser and your observability tooling already exist for that shape, and every one of them is code you do not write or test. The compatible endpoint also gets you the <think>-block parsing and the tool-call parsing described in the tool-call format page without implementing either.

Reach for the native endpoint when a parameter has no OpenAI equivalent. That is where the vendor-specific controls live, and it is worth checking the current parameter list before concluding something is impossible rather than assuming the compatible surface is the whole API.

Related