The repetition_penalty Parameter in Qwen and How It Interacts With Sampling

repetition_penalty is not a penalty in the additive sense that OpenAI’s frequency_penalty is. It is a multiplicative rescaling of the logit, it treats positive and negative logits differently, and it runs before truncation sampling — which is why a value that looks small can remove a token from consideration entirely.

The formula

The parameter comes from the CTRL paper (Keskar et al., 2019), and the implementation every Qwen serving stack inherits is the one in Hugging Face transformers’ RepetitionPenaltyLogitsProcessor. For each token t that already appears in the sequence, with logit x and penalty p:

if x > 0:  x' = x / p
else:      x' = x * p

tokens not yet seen are left unchanged
p = 1.0 disables the processor entirely

A worked example, with p = 1.1 and three candidate logits:

token       logit   seen?   after penalty
" the"       8.0     yes     8.0 / 1.1  =  7.27
" a"         7.4     no      7.4        =  7.40
" that"     -2.0     yes    -2.0 * 1.1  = -2.20

Note what happened to the ranking: " the" was the top-scoring token before the penalty and is second after it. The penalty did not merely reduce its probability, it changed the order — and order is what every downstream truncation step reads.

Source for the implementation: the transformers generation documentation, and the original description in Keskar et al., “CTRL” (2019).

The sign asymmetry

The branch on x > 0 is the part that surprises people, and it is deliberate. If the rule were simply “divide by p”, then a token with a logit of -2.0 would become -1.82 — closer to zero, and therefore more likely. Dividing a negative number by a number greater than one raises it. The penalty would reward repetition of tokens the model already disliked.

So negative logits are multiplied instead, pushing them further down. The behaviour is consistent — repeated tokens always get worse — but the magnitude is not symmetric, and the practical consequence shows up at large penalty values. At p = 1.5, a logit of 8.0 drops by 2.67 while a logit of -8.0 drops by 4.0. Tokens deep in the tail are pushed away far harder than the leaders are, which flattens the head of the distribution while gutting the tail. Above roughly 1.2 this reliably produces the failure people describe as the model “getting weird”: it will not reuse a word it needs, so it reaches for a worse synonym, and in code or structured output that means broken syntax, because the closing brace is a token it has used before.

Where it sits in the sampling pipeline

The penalty is a logits processor, and processors run before the warpers that implement temperature, top-k and top-p. The order for a Qwen generation is:

raw logits
  -> repetition_penalty      (rescale seen tokens)
  -> temperature             (divide all logits)
  -> top_k                   (keep k highest)
  -> top_p                   (keep smallest set summing to p)
  -> softmax and sample

This ordering is the reason the parameter interacts with the others rather than composing cleanly with them. Because the penalty reorders before top_p truncates, a token can be pushed out of the nucleus — not made less likely, but made ineligible. With top_p = 0.8 and a distribution where one token holds most of the mass, penalising that token can admit several new tokens into the nucleus at once, which is a much larger change in behaviour than the logit shift suggests.

It also means repetition_penalty and low temperature fight each other. Temperature near zero exists to make the top-ranked token win; the penalty exists to demote the top-ranked token when it has been seen. Setting temperature=0 with repetition_penalty=1.2 gives you deterministic output that systematically avoids its own vocabulary, which is rarely what anyone wanted.

Order also explains why the penalty is scale-dependent in a way temperature is not. Temperature divides every logit by the same number, so it changes how peaked the distribution is without changing the ranking at all. The penalty changes only some logits, and by a multiplicative factor, so its effect depends on how large the logits happen to be at that step. The same p applied where the top logit is 15 removes far more absolute score than where the top logit is 3. There is no single value that means “discourage repetition by this much” across positions.

Compare that with the additive penalties in the OpenAI family, which Model Studio also exposes for the commercial Qwen endpoints:

repetition_penalty   x' = x / p   (or x * p if x <= 0)   multiplicative
presence_penalty     x' = x - c   if seen at all               additive, flat
frequency_penalty    x' = x - c*n if seen n times              additive, scaled

The additive forms subtract a fixed number of logits regardless of scale, so their effect is predictable across positions, and frequency_penalty can distinguish a word used once from a word used eleven times — a distinction repetition_penalty cannot make, since it is a set membership test. Where both are available, the additive ones are the easier parameters to reason about.

It penalises your prompt too

“Tokens that already appear in the sequence” means the whole sequence, prompt included. The processor is given the full input_ids, so every token in your system prompt, your documents, and the earlier turns of the conversation is already marked as seen before the first output token is generated.

This is the single most damaging misunderstanding of the parameter, because it makes it actively harmful for exactly the tasks people most often apply it to. Summarisation, translation, extraction and retrieval-augmented answering all require the model to reuse words from the input. A penalty of 1.15 on a summarisation task tells the model to avoid the vocabulary of the document it is summarising.

Long contexts make it worse. A 20,000-token document has marked a large fraction of the plausible vocabulary as seen, so the penalty applies almost everywhere and stops discriminating between genuine repetition and ordinary language.

Choosing a value

• Start at 1.0. Disabled is the correct default for most Qwen workloads. Qwen3’s degenerate-repetition guidance points at sampling settings and at presence_penalty, not at large repetition penalties.
• 1.05 to 1.1 for open-ended prose that loops. This is the range where the effect is a nudge rather than a constraint.
• Leave it at 1.0 for code, JSON and tool calls. Structured output is repetitive by construction. Penalising a repeated "name" key produces invalid output, and the failure looks like a model problem rather than a parameter one.
• Prefer the additive penalties if they are available. presence_penalty and frequency_penalty subtract a fixed amount rather than rescaling, so they do not have the sign asymmetry and they scale with how often a token appeared rather than merely whether it did. Model Studio exposes presence_penalty for the commercial Qwen endpoints.
• If the model never stops, this is the wrong lever. A generation that runs to the ceiling is usually a stop-token or template problem — see when Qwen does not stop generating — and turning up the repetition penalty to mask it degrades every other request.