Migrating an Internal Prompt Injection Bug Bounty Scope

A bounty scope document is a contract about what counts as a finding. When the model underneath changes, the document does not change, but a quarter of what it asserts stops being true — and the people it misleads are your own triagers, who will close valid reports as known behaviour.

What the migration silently invalidated

Most internal injection-bounty scopes contain four kinds of statement, and they age at very different rates. Statements about your infrastructure (which hosts are in scope, which environments testers may touch) survive a model change entirely. Statements about your application’s intended behaviour (the assistant must never send an email without confirmation) survive too, because they are product requirements.

The two that do not survive are statements naming a specific model or endpoint, and statements asserting what the model will or will not do. The first kind is obvious and someone usually catches it: a scope that lists a model string as the target of testing needs the new string. The second kind is invisible, because it reads like policy rather than like an observation, and nothing about the migration flags it.

The known-safe list is the dangerous part

Almost every mature scope grows an out-of-scope section that accumulated one entry per closed report. Entries in it read like “the assistant will not reveal its system prompt when asked directly; direct extraction attempts are out of scope”, or “the model declines requests to enumerate other tenants’ identifiers, so we do not pay for that class”.

Every one of those is an empirical claim about a model that is no longer serving the traffic. They were true when written, they were verified once, and the verification does not transfer. A researcher who reports direct system-prompt extraction the week after cutover gets an automated “known behaviour, see scope section 4” and the finding dies in the queue — the exact failure mode a bounty program exists to prevent.

The fix is a rule rather than an edit: an out-of-scope entry that asserts model behaviour must carry the date and the model it was verified against, and must be treated as expired the moment either changes. Entries whose justification is architectural — the model has no credential to leak because credentials never enter its context — carry no expiry, because they do not depend on the model choosing well. That distinction, between an entry justified by behaviour and one justified by architecture, is worth marking in the document itself.

Scope by trust boundary, not by model

The durable formulation defines findings in terms of your own trust boundary and says nothing about how the model behaves. Three surfaces make up that boundary in almost every deployment:

• The tool surface. Any injected instruction that causes a tool call the user did not ask for is a finding, whatever the model, because the impact is your side effect and not the model’s wording. This is the class most worth paying for and the least sensitive to a model swap.
• The retrieval corpus. Content that any user can put into a place the assistant later reads — a support ticket, a shared document, a filename — is an injection vector you own. Whether a given payload works depends on the model; whether the vector exists does not.
• The output sink. Wherever model output is rendered, executed or forwarded without escaping — a markdown renderer that follows links, a shell, a downstream API call. The classic image-tag exfiltration finding lives here and is a rendering bug wearing a model costume.

A scope written this way needs no edits when the provider changes, because none of its clauses mention one. What changes is which payloads reach the boundary, and that is a question for your regression corpus, not for the scope document. Keeping the two apart is most of the work.

Whose bug is it

The migration also moves a line that scope documents rarely state explicitly: which reports belong to you and which belong to the provider. A payload that makes the model produce disallowed text with no effect on your system is a model-behaviour report; it belongs in the provider’s own reporting channel, and yours should say so and give the address. A payload that makes the model do something to your data or your users is yours regardless of how it was phrased.

The line is easier to hold if you state it as a question about impact rather than about content. Did the payload change what your system did, or only what the model said? A payload that produces an offensive paragraph and nothing else changed no state, touched no data and crossed none of the three surfaces above; it is a model-behaviour report. A payload that produced the same paragraph and also caused a record to be written is yours, and the paragraph is incidental to it.

Both providers run their own intake for model-behaviour reports, and the address is the thing to update — a scope that routes researchers to the previous vendor’s form after cutover wastes their time and yours. State the routing rule in one sentence and give the current link; do not paraphrase the provider’s own program terms, which change and are not yours to summarise.

Re-baseline before you re-open

The sequencing that avoids the queue-poisoning failure is: freeze submissions or hold triage during the cutover window, run your existing injection corpus against the new model and record pass and fail per case, then rewrite the expired known-safe entries from that result, and only then re-open. The corpus run is the same artefact used for re-baselining injection tests, so the work is shared rather than duplicated.

Expect the result to be mixed rather than uniformly better or worse. Some payloads that used to work stop working; some that never worked start to. Neither direction is a reason to change the scope document’s clauses — both are reasons to change the known-safe list, which is now dated and provisional by construction. Publish the re-baselining date at the top of the scope so a researcher reading it knows how fresh the behavioural claims are, and put the next review on the calendar alongside the payout structure review rather than leaving it to the next incident to trigger.