Rewriting Prompts That Do Not Destroy Meaning

Ask a model to rewrite a paragraph and it will improve it. That is the problem. The improvement is usually a reason, a benefit or a qualifier that was not in the original, and it arrives in the same register as everything else, so nobody catches it.

What a rewrite adds when nobody is looking

Rewriting is generation conditioned on a source, and the conditioning is soft. The model is producing fluent prose in a style, and fluent prose in most styles has a shape — claim, reason, benefit — that the source may not fill. When a slot is empty, it gets filled.

source:
  "The API returns 429 when you exceed 60 requests a minute."

rewrite for a marketing page:
  "Our API is built for scale: if you go beyond 60 requests a minute, you'll
   get a friendly 429 so your integration stays stable and your costs stay
   predictable."
                    ^ not in source   ^ not in source        ^ not in source

Three inventions, none of them a lie the writer would have told, all of them now in your documentation. The prompt below prevents them by making “add nothing” the highest-priority constraint and by giving the model somewhere to report a constraint it could not meet, rather than quietly resolving the conflict in favour of style.

The prompt

Rewrite the text in <source>.

Constraints, in priority order. Where two conflict, the lower-numbered one
wins and you report the conflict.

1. Add no factual content. Every claim in your output must also be in
   <source>. No reasons, no benefits, no examples, no qualifiers, no
   consequences, no "which means", no "so that".
2. Preserve every number, name, date, product name, URL, error code and
   quoted phrase exactly as written in <source>.
3. Preserve every claim in <source> unless <drop> lists it. Do not merge two
   claims into one that says less than both.
4. Meet this target: {{target}}
   (for example: "at most 90 words"; "second person"; "no sentence over 20
   words"; "for a reader with no technical background")
5. Match this register: {{register}}

If 4 or 5 cannot be met without breaking 1, 2 or 3, meet 1 to 3 and record
what you could not do in "unmet".

Return JSON:
{"rewrite": "...",
 "claims_kept": ["<one line per claim from <source> that survives>"],
 "claims_dropped": [{"claim": "...", "why": "listed in <drop>" | "..."}],
 "unmet": ["<constraint number and what stopped it>"]}

<source>
{{source}}
</source>

<drop>
{{claims_to_remove}}
</drop>

Why the constraints are numbered

Unnumbered constraints are a set of simultaneous demands, and a rewrite almost always makes at least two of them impossible together — “at most 90 words” and “keep every claim” is the usual pair. Faced with an impossible set, a model satisfies the ones that are easiest to satisfy visibly. Length is visible. Claim preservation is not. So the unnumbered version silently drops a claim to hit the word count.

Numbering does not make the conflict go away; it decides who wins, and it puts the loss on the record. The unmet field is the half people leave off. Without it the model still has to choose, and you still never find out. With it you get a queue of “could not hit 90 words without dropping the refund window”, which is a decision a human can make in five seconds.

Constraint 1 is deliberately a list of the specific additions rather than a general prohibition. “Add nothing” alone is read as “do not add new topics”; a model does not classify “so your integration stays stable” as new content, because it feels like elaboration. Naming which means and so that as banned constructions is more effective than the abstract rule, for the same reason that a negative instruction works better when it names the thing.

claims_kept is not for the reader. It is the input to the check below, and asking for it in the same call is cheaper than deriving it in a second one — though you should still not trust it, because it is produced by the same call that produced the rewrite.

The diff check, in two stages

Stage one is deterministic and free. Stage two costs a call and is the one that catches inventions.

import re

TOKEN = re.compile(r"\d[\d,.:]*\d|\d|[A-Z][A-Za-z0-9_.-]{2,}|https?://\S+")

def surface_diff(source: str, rewrite: str) -> dict:
    """Numbers, capitalised names, codes and URLs that appear in one and not
    the other. Cheap, high-precision, catches constraint 2."""
    s = set(TOKEN.findall(source))
    r = set(TOKEN.findall(rewrite))
    return {"invented": sorted(r - s), "lost": sorted(s - r)}

invented is the alarming list: a number or a product name in the rewrite that is nowhere in the source is a hard failure with no judgement involved. lost is advisory — a rewrite is allowed to drop things, and constraint 3 says which — so treat it as a list to reconcile against claims_dropped rather than as an error.

This will produce false positives on capitalised sentence-initial words. Either restrict the name pattern to tokens that appear mid-sentence, or accept the noise; at these volumes a short review list is cheaper than a clever regex.

The entailment prompt

Stage two splits the rewrite into claims and asks whether the source supports each one. The rule that makes it work is in the second paragraph.

Split <rewrite> into atomic claims, then judge each against <source>.

An atomic claim is one assertion. "The API returns 429 above 60 requests a
minute, which keeps costs predictable" is two claims, not one.

For each claim return:
{"claim": "...",
 "verdict": "supported" | "unsupported" | "overstated" | "weakened",
 "quote": "<verbatim span from <source>, or null>"}

A verdict of "supported" REQUIRES a quote from <source> that establishes the
claim on its own. If you cannot produce that span, the verdict is
"unsupported". Do not reason your way to support.

"overstated": <source> says something weaker (it says "may", the claim says
"will"; it says "some", the claim says "all").
"weakened": <source> says something stronger and the claim hedges it away.

Judge support only. Do not judge whether the claim is true in the world, and
do not judge style, tone or length.

<source>{{source}}</source>
<rewrite>{{rewrite}}</rewrite>

“Supported requires a quote” is the mechanical part. It converts a judgement — is this entailed — into the production of an object you can verify with a substring test. A judge that can say “supported” without pointing at anything will say it for claims that merely sound consistent with the source, which is exactly the failure class here. Verify the quotes yourself with the same canon function used in the extraction recipe; a “supported” verdict whose quote is not in the source is worse than an “unsupported” one.

Splitting overstated and weakened out of unsupported matters because they need different handling. Overstatement is a compliance problem in marketing and a safety problem in anything clinical or financial. Weakening is usually harmless and occasionally the point of the rewrite. Collapsing them into one verdict means you cannot set different thresholds, which you will want to.

The 429 paragraph, run through

source:
  "The API returns 429 when you exceed 60 requests a minute. The limit is per
   API key. Retry-After gives the number of seconds to wait."

target:   "for a getting-started page, at most 40 words, second person"
drop:     (nothing)

-- the rewrite --------------------------------------------------------------
{"rewrite": "Go over 60 requests a minute and the API returns 429. The limit
             applies per API key. The Retry-After header tells you how many
             seconds to wait.",
 "claims_kept": ["429 above 60 requests a minute",
                 "limit is per API key",
                 "Retry-After gives seconds to wait"],
 "claims_dropped": [],
 "unmet": []}

-- surface_diff -------------------------------------------------------------
{"invented": [], "lost": []}

-- entailment ---------------------------------------------------------------
[{"claim": "The API returns 429 above 60 requests a minute",
  "verdict": "supported",
  "quote": "The API returns 429 when you exceed 60 requests a minute."},
 {"claim": "The limit applies per API key",
  "verdict": "supported", "quote": "The limit is per API key."},
 {"claim": "The Retry-After header tells you how many seconds to wait",
  "verdict": "supported",
  "quote": "Retry-After gives the number of seconds to wait."}]

Now the same source with a target that cannot be met honestly, which is where the priority order earns its place:

target:   "for a marketing page, at most 40 words, warm and reassuring"

{"rewrite": "The API returns 429 once you pass 60 requests a minute. The limit
             is per API key, and Retry-After tells you how many seconds to
             wait.",
 "claims_kept": ["429 above 60 requests a minute", "limit is per API key",
                 "Retry-After gives seconds to wait"],
 "claims_dropped": [],
 "unmet": ["5: 'warm and reassuring' would require adding a reason the rate
            limit exists or a benefit it provides. Neither is in <source>."]}

That unmet entry is the output the whole prompt exists to produce. The model has correctly identified that the register it was asked for is unreachable from the facts it was given, and said so instead of manufacturing the missing benefit. Somebody reading the queue can then do the only thing that actually fixes it: supply the reason, as a fact, in the source.

Compare with the failure at the top of this page, which is what the same request produces without constraint 1. The three inventions there would all come back from the entailment call as unsupported with quote: null — which is the check working, but a caught invention is still a wasted call, and the ordering is what prevents it rather than detects it.

When it stops working

• unmet is always empty. On real work it should fire regularly — genuine constraint conflicts are common. An empty field for a week means the model has stopped reporting rather than stopped conflicting, and something is being dropped silently again.
• invented starts catching things the entailment call passes. The judge has become lenient. Check that the quote requirement survived your last prompt edit; it is the line most often lost when somebody shortens the prompt.
• Rewrites get shorter than the target. Usually constraint 1 winning too hard — the model drops claims to avoid any risk of elaboration. Loosen by naming what it may add: nothing, but it may reorder and it may split a sentence.