frequency_penalty and presence_penalty in the OpenAI API: What Each One Changes

Both parameters discourage repetition and they do it by different arithmetic on the same quantity. OpenAI publishes the formula, which means the effect of any particular value is not a matter of feel — you can compute it.

The formula, both penalties in one line

OpenAI’s API reference gives both penalties as a single adjustment applied to the logits before sampling. Writing mu[j] for the logit of token j and c[j] for how many times that token has already appeared in the text so far:

mu[j]  →  mu[j]  −  c[j] × alpha_frequency  −  float(c[j] > 0) × alpha_presence

alpha_frequency = frequency_penalty   range −2.0 … 2.0   default 0
alpha_presence  = presence_penalty    range −2.0 … 2.0   default 0

Three things follow directly from that line and they are the whole subject.

• The adjustment is subtractive on logits, which means it is multiplicative on the unnormalised probabilities. Subtracting x from a logit multiplies that token’s weight by e to the power of −x.
• It happens before the softmax, and therefore before temperature and top_p do their work. Penalised tokens can be pushed out of the nucleus entirely as a second-order effect.
• Negative values are legal and invert the effect, increasing the probability of tokens that have already appeared. That is occasionally what you want for a highly repetitive format, and at large negative values it produces the degenerate loop the positive values exist to prevent.

Flat versus proportional

The difference between the two is entirely in the second factor of each term.

• frequency_penalty is multiplied by c[j], the raw count. It grows without limit: a token seen twenty times is penalised twenty times as hard as one seen once.
• presence_penalty is multiplied by float(c[j] > 0), which is 1 or 0. It is a flat, one-off charge for having appeared at all, identical whether the token appeared once or a hundred times.

So they answer different questions. presence_penalty pushes the model towards new tokens — it widens vocabulary, and its effect is fully applied the instant a token first appears. frequency_penalty pushes against over-use, and it ramps: it barely touches a token on its second appearance and becomes severe by its tenth. If your problem is that the model loops on one phrase, frequency is the one that escalates against a loop. If your problem is that the whole answer draws on a narrow vocabulary, presence is the one that costs nothing extra for the eleventh repeat but taxes the first.

What 0.5 does to a token seen eight times

Take a token that has already occurred eight times — the word the in a paragraph of ordinary prose is a realistic count. Set each penalty to 0.5 in turn and compute the logit adjustment and the resulting multiplier on the token’s unnormalised weight.

c[j] = 8

frequency_penalty = 0.5
  logit change  = −(8 × 0.5)          = −4.0
  weight factor = exp(−4.0)           = 0.0183     ≈ 55× less likely

presence_penalty = 0.5
  logit change  = −(1 × 0.5)          = −0.5
  weight factor = exp(−0.5)           = 0.6065     ≈ 1.65× less likely

both at 0.5
  logit change  = −4.0 − 0.5          = −4.5
  weight factor = exp(−4.5)           = 0.0111     ≈ 90× less likely

The gap between the two at the same nominal value is the finding here. A frequency_penalty of 0.5 on a token with eight occurrences is not a gentle nudge — it is a factor of fifty-five against that token relative to its unpenalised competitors, which on most distributions removes it from contention entirely. The same 0.5 as a presence penalty is a factor of 1.65, a mild preference.

Follow the frequency case one step further to see why the parameter escalates so hard. The multiplier for a count of c at frequency_penalty f is exp(−c × f), which decays geometrically in the count:

frequency_penalty = 0.5

c =  1   factor 0.607
c =  2   factor 0.368
c =  4   factor 0.135
c =  8   factor 0.018
c = 16   factor 0.0003
c = 32   factor 0.0000001

By the sixteenth occurrence the token is effectively banned. That is the mechanism behind the classic symptom of an over-penalised generation: the text starts normally, degrades into odd synonym choices, and then breaks grammatically — because the tokens it has run out of are the common ones.

They penalise tokens, not ideas

The counts are over tokens as the tokeniser produced them, which is the constraint that decides what these parameters can and cannot fix.

Function words are single frequent tokens. In any English text of a few hundred words, the, of, to, a space, a comma and a newline will all have high counts long before any content word does. A frequency penalty large enough to break a repeated phrase has already applied a much larger penalty to the grammar, because the grammar tokens are the ones with the counts. This is why high frequency_penalty values produce text that reads as though it is avoiding something.

Structured output is the sharp case. JSON repeats {, ", : and , constantly and correctly; a list of twenty objects with the same keys is supposed to repeat every key. Applying a repetition penalty to a request with a JSON schema response format is arguing directly against the format you asked for. The safe default for any machine-readable output is 0 for both.

Nor do they help with semantic repetition. A model that says the same thing three times in three different sentences has repeated an idea, not a token sequence, and nothing in the formula above can see that. That is a prompt problem.

A related trap when moving code between APIs: the open-weights ecosystem’s repetition_penalty is a different formula — it divides or multiplies the logit rather than subtracting from it, with a neutral value of 1.0 rather than 0.0. Passing an OpenAI-shaped value into it does not mean the same thing, and a 0.0 there is not neutral at all.

Choosing a value

The range is −2.0 to 2.0 and the useful positive range is far narrower than that range suggests. From the arithmetic above:

• 0 for anything structured, anything short, and anything where correctness beats variety. This is the default and it is the right default.
• 0.1 to 0.5 presence_penalty for long prose where you want broader vocabulary. The flat charge does not compound, so the failure mode is mild.
• 0.1 to 0.3 frequency_penalty for a model that visibly loops. Above roughly 0.5 the compounding is doing more to your function words than to your loop, and the fix is usually the prompt or the stop sequence rather than a larger number.
• Both at once, cautiously. Unlike temperature and top_p these do compose in a way you can reason about — the two terms simply add in log space — but the frequency term dominates as soon as counts are non-trivial, so a presence penalty set alongside a frequency penalty of any size is mostly decorative.

Whatever you pick, evaluate it on output length that matches production. These parameters have almost no effect on a 30-token answer, because nothing has a count yet, and a large effect on a 1,500-token one. A value tuned on short samples will surprise you in production, and the direction of the surprise is always the same.

If what you actually want is to suppress one specific token rather than repetition in general, neither penalty is the right instrument — both are indiscriminate by construction, because the formula has no term for which token is being penalised. The parameter for that is logit_bias, a map from token id to a bias between −100 and 100 applied to that token’s logit at every step, where −100 makes a token effectively unreachable. It acts on the same quantity as the penalties and at the same point in the pipeline, and it is surgical where they are blunt. The cost is that you must supply token ids from the model’s own tokeniser rather than strings, and the ids differ between tokeniser generations — a bias map built against one vocabulary is meaningless on another and will quietly penalise unrelated tokens rather than erroring.

The same reasoning applies in reverse to the most common misuse of these parameters, which is reaching for them when the model repeats because the prompt made repetition the likely continuation. A prompt that shows three examples all beginning “Certainly! Here is” has taught the model that this is how answers start, and no value of either penalty is a substitute for deleting that pattern from the prompt. Penalties fight the distribution; the prompt shapes it, and shaping is cheaper than fighting.