Qwen3's Thinking Mode: The <think> Tag Convention

Qwen3 is a hybrid: the same weights answer directly or reason first, and the reasoning arrives in the ordinary content stream wrapped in a <think> block. If you are reading the raw completion, that block is your problem to handle — and handling it wrong in conversation history is the mistake that degrades the model over a long session.

The raw output

Here is the shape of a Qwen3 completion in thinking mode, abbreviated in the middle but structurally exact:

<think>
The user is asking for the capital of Portugal. This is a factual
question with no ambiguity. Lisbon has been the capital since 1255,
when it replaced Coimbra. No need to hedge or ask for clarification.
</think>

Lisbon.

The important structural facts about that block:

• It is content. There is no separate field in the raw completion, no different token stream, nothing distinguishing it from the answer except the delimiters.
• <think> and </think> are single tokens in Qwen3’s vocabulary, registered in added_tokens.json. Resolve their ids with tok.convert_tokens_to_ids("<think>") rather than hard-coding them — they are stable within a generation but not a promise across one.
• The block comes first, exactly once, and the answer follows it. There is no interleaving of reasoning and answer in Qwen3’s convention.
• The reasoning is billed as output tokens like any other, which is why a thinking-mode answer of one word can cost a thousand tokens.

The behaviour and its parameters are documented on the Qwen3 model cards — for example Qwen/Qwen3-8B on Hugging Face — and in the Qwen3 technical report (arXiv:2505.09388, May 2025).

Parsing it out

The naive approach is a regular expression over the finished string, and for a non-streaming call it is genuinely adequate:

import re

THINK = re.compile(r"^\s*<think>(.*?)</think>\s*", re.DOTALL)

def split_thinking(text: str) -> tuple[str, str]:
    m = THINK.match(text)
    if not m:
        return "", text.strip()
    return m.group(1).strip(), text[m.end():].strip()

Anchoring at the start with match rather than searching anywhere matters: a model discussing thinking tags, or echoing a document that contains them, can put the literal string in the middle of an answer, and a global search would treat that as a reasoning block and delete everything before it.

The token-level approach is more robust and is what you want if you are working with ids rather than text: find the id of </think> in the output token list and split there. It cannot be confused by content because a literal </think> written by the model as text tokenises differently from the special token.

On the streaming path neither works, because you do not have the whole string. You need a small state machine: start in the “in thinking” state if the first token is <think>, switch to “answering” on </think>, and route deltas to different sinks accordingly. The consumer of a streaming endpoint usually wants to show the reasoning in a collapsed panel and the answer in the main pane, and those are two sinks. Alibaba’s hosted API sidesteps this by putting the reasoning in a separate reasoning_content field on each chunk — see the streaming formats page for what that looks like on the wire.

Why the block must not go back in

When you append the assistant’s reply to the conversation for the next turn, append the answer only. Qwen’s documentation is explicit that thinking content should be excluded from history, and the reason is mechanical rather than stylistic.

The post-training data taught the model that a completed assistant turn in history looks like an answer, without a reasoning block. Feeding reasoning back puts the model in a context it did not see during training, and the observed consequences are the ones you would predict: it starts reasoning about its own previous reasoning, blocks grow turn over turn, and by turn five a large fraction of the context is deliberation about deliberation.

There is a cost argument on top of the quality one. Reasoning blocks are the largest part of a thinking-mode response, so keeping them in history means every subsequent turn re-reads all of them as input tokens. On a long session that is the dominant term in the bill, and it buys nothing.

Note that this is the opposite of what some other families require — extended-thinking implementations that sign their reasoning blocks expect them returned intact. There is no universal rule here; it is per-family behaviour, and DeepSeek-R1’s handling of the same tags is worth reading alongside this for the contrast.

The empty block in non-thinking mode

Switch thinking off and the block does not vanish — it arrives empty:

<think>

</think>

Lisbon.

This is the chat template’s doing, not the model’s. When enable_thinking=False, the template pre-fills a closed, empty think block into the assistant turn so that generation begins after it. It is a prefill trick: the model is placed in the state it would be in having already finished reasoning, which is a far more reliable way of suppressing the behaviour than instructing it not to reason.

The consequence for your parser is that it must handle an empty block without treating the response as malformed, and must not decide “thinking was on” merely from the presence of the tags. The parameter itself, including the /think and /no_think soft switches you can put in a user message to flip modes mid-conversation, is covered in the enable_thinking page.

Sampling parameters differ by mode

Qwen’s model cards publish different recommended sampling settings for the two modes, which is unusual and worth respecting because the same weights behave differently under them. As documented on the Qwen3 cards: thinking mode uses temperature 0.6, top-p 0.95, top-k 20 and min-p 0; non-thinking mode uses temperature 0.7, top-p 0.8, top-k 20 and min-p 0.

The cards also warn specifically against greedy decoding in thinking mode. The mechanism is worth understanding rather than obeying blindly: a long reasoning chain sampled at temperature 0 is a long sequence of locally-maximal tokens, and locally-maximal sequences are exactly where repetition loops live. The model gets into a phrase whose highest-probability continuation returns to the same phrase, and with no randomness there is nothing to break the cycle. It will then generate until it hits your output cap. If you have set temperature 0 for reproducibility and you are seeing pathological repetition in reasoning, that is the cause.