Mapping Frequency and Presence Penalty Parameters Between APIs

These two parameters are usually explained as “reduce repetition”, which is true and useless when you are deciding what to do on an API that does not have them. Both are one subtraction applied to the model’s scores, and once you have the subtraction, the migration question answers itself.

One formula, two coefficients

Before a token is sampled, the model produces a score for every token in the vocabulary. The penalties modify those scores using a count of what has already appeared in the sequence. In the form OpenAI documents for its Chat Completions API, the adjusted score for token t is:

adjusted[t] = logit[t]
              - presence_penalty  * (count[t] > 0 ? 1 : 0)
              - frequency_penalty * count[t]

count[t]  how many times token t already appears in the
          prompt-plus-completion so far

Everything else follows from those two lines. The presence term is a one-off toll: a token pays it the moment it has appeared even once, and pays exactly the same amount if it has appeared forty times. The frequency term is proportional: each additional occurrence subtracts another frequency_penalty from the score. Both accept the range -2.0 to 2.0 and both default to 0 on that API, and negative values invert them — a negative frequency penalty rewards repetition, which is occasionally what you want for a model that keeps paraphrasing a term you need verbatim.

One property that surprises people: the penalty is applied to tokens, not to words or concepts. Penalising a token does not stop the model expressing the same idea with a different word, and it does penalise tokens that appear inside completely unrelated words. Structural tokens are the common casualty. Push the frequency penalty up on a model emitting JSON and you are penalising the quote, brace and comma tokens it must repeat to produce valid output; this is a well-understood way to break structured generation, which is why strict schema modes and penalties do not belong in the same request.

Where the two diverge in practice

Because the presence term saturates after the first occurrence, it shifts the model toward new material without punishing a term it must keep using. Because the frequency term grows without bound, it escalates: a long completion that legitimately repeats a product name twenty times has that name pushed steadily further down the distribution until the model reaches for a synonym, a pronoun, or a different topic altogether.

That escalation is the reason a high frequency penalty degrades long outputs more than short ones. It is also why the two parameters are not interchangeable at any setting: their effect on a token that has appeared once is identical in shape and their effect on a token that has appeared fifteen times differs by a factor of fifteen. Any mapping table that treats them as two dials for the same thing will produce bad conversions in exactly the cases that matter.

Which API surfaces express them

The names frequency_penalty and presence_penalty originate in OpenAI’s Chat Completions API and are carried by the large number of servers that implement an OpenAI-compatible endpoint — local runtimes, open-model hosts and inference vendors that advertise drop-in compatibility. Compatibility here is a claim about field names, not about behaviour: an OpenAI-compatible server may accept both fields and implement one, or implement both with a different counting window, or accept and silently ignore them. Assume nothing from acceptance of the field.

Anthropic’s Messages API has neither. Its documented request body is model, messages, max_tokens, system, temperature, top_p, top_k, stop_sequences, stream, tools, tool_choice, thinking, metadata and service_tier, per Anthropic’s Messages API reference, with temperature documented over 0.0 to 1.0 rather than 0 to 2. There is no repetition control of any kind in that list, which makes it the canonical case for the rest of this page.

Google’s Gemini API puts sampling controls in a nested generationConfig object using camelCase names — temperature, topP, topK, maxOutputTokens, stopSequences and candidateCount among them. Read the current GenerationConfig reference for that API to see whether penalty fields are exposed on the model you are targeting rather than assuming from another provider’s surface; this page does not assert an identifier it could not confirm.

What is not a substitute

The instinct on a target without penalties is to reach for temperature or top-p. Neither does the job, and the formula shows why. Temperature divides all the scores by a constant before the softmax: it flattens or sharpens the whole distribution uniformly and has no term that depends on what has already been generated. Top-p and top-k truncate the candidate set by rank, again with no history term. A repeated token that the model strongly prefers is still the top-ranked token after any amount of temperature or truncation, because none of those operations know it was repeated. Raising temperature to break a repetition loop works by making every choice noisier, which trades the loop for a general loss of precision.

Some APIs offer a per-token bias map instead, which lets you subtract a fixed amount from named tokens. That is a genuine relative of the presence penalty — a constant subtraction — but it is static: you set it before the request and it cannot respond to what the model generates. It handles “never say this word” well and “stop saying this word so often” not at all.

Handling the gap in an adapter

Three approaches actually work on a target with no penalty support, in increasing order of cost.

1. Drop it and check. If the penalty was set to a small value years ago and nobody can say why, run your eval set with and without it on the old provider first. A surprising number of these settings do nothing measurable, and discovering that removes the migration problem entirely.
2. Move the constraint into the prompt and the stop list. Repetition that the penalty was suppressing is often structural — restating the question, re-listing the same three points — and an explicit instruction plus a stop sequence handles the structural case more reliably than a logit adjustment ever did.
3. Detect and regenerate. Post-process the completion for the repetition you care about — an n-gram repeated beyond a threshold, a sentence appearing twice — and re-request with a stronger instruction when it trips. This costs a second call on the small fraction of responses that fail, and unlike a penalty it acts on the thing you actually object to rather than on token counts.

Whichever you choose, do not have your adapter forward the parameter hopefully. An unknown field is rejected outright by some APIs and ignored by others, and the second is worse: the request succeeds, the setting does nothing, and the behaviour change is attributed to the model. Make the adapter explicit about what it dropped, and log it.