Handling API Key Rotation During a Provider Migration

Every step in a key rotation is reversible except one. Order the work so that the irreversible step — revoking the old key — happens last, and is triggered by evidence that nothing is using it rather than by a date in a ticket.

What a key actually scopes

Before rotating anything, find out what the key you hold is attached to, because the answer decides whether a second key is a safe no-op or a behaviour change.

• Rate limits. Whether limits are enforced per key or per organisation is the question that matters most. If limits are per organisation, adding a second key gives you no extra throughput and the two keys compete. If they are per key, a new key may start with different limits from the one you have been running on for a year — and a freshly created key on a lower default tier will start returning 429s under a load the old key handled. Read the current rate-limit documentation for the provider rather than assuming, and rate limits explained covers how the two limit dimensions interact.
• Spend controls. Budgets and hard caps are often set per project or per key. A new key outside the existing budget can either bypass a limit you rely on, or hit a zero default.
• Project or workspace scoping. Where a provider has projects, a key belongs to one, and resources created under one project — fine-tunes, uploaded files, batch jobs — may not be visible to a key from another. This is the one that breaks asynchronous work: a batch submitted with the old key and polled with the new one can come back as not found.
• Header shape. Across providers this is not even consistent in form. Some take an Authorization: Bearer header, some a vendor-specific header name, some accept a query parameter. If your HTTP layer has one place that attaches credentials, it needs to become provider-aware rather than key-aware.

The overlap window

The invariant to hold is that at every instant, every running process holds at least one credential that works for the provider it is about to call. Everything else follows from it.

1. Teach the code to hold more than one credential before you create any. This is the step people skip, and it is the one that makes the rest safe. Credential resolution becomes a lookup keyed by provider, returning the current secret; the deploy that introduces it changes no behaviour at all and can be rolled back freely.
2. Create the new credential and store it under a new name.Never overwrite the old value in the secret store. Overwriting means the rollback is “paste the old secret back in”, which requires still having it.
3. Verify the new credential out of band, with the check in the next section. Do not verify it by sending production traffic at it.
4. Shift a small share of traffic and watch the authentication error rate specifically, separately from your general error rate. Auth failures are a distinct class and should have their own alert; buried in a total error rate, a 2% shift failing entirely is invisible.
5. Ramp to 100% and leave the old credential valid. The migration is now functionally done and completely reversible.
6. Revoke only after the old key has been observably idle for longer than your longest-lived process. See below.

Two properties of that ordering are worth naming. Adding a credential is reversible; removing one is not. And the deploy that adds multi-key support is separate from the deploy that uses it, so if something breaks you know which change caused it.

Verifying a key without spending much

“Does this key work” should be answerable in one cheap call that does not run inference. Where the provider offers a metadata endpoint — a models list, an account or usage endpoint — that is the right probe: it requires authentication, it costs nothing, and its response tells you whether the key is valid and often what it can see. Where there is no such endpoint, a completion with a one-token output cap and a trivial prompt is close enough to free.

Check three things, not one. That the call succeeds. That the response shows the account or project you expected, if the endpoint reveals it — a valid key pointing at the wrong project is a real and confusing failure. And that a deliberately wrong key fails, which proves your probe is actually authenticating rather than hitting an unauthenticated path.

Make the probe part of deployment rather than a manual step. A startup check that resolves every configured credential and probes each one turns “the key was never set in staging” from a 3am incident into a failed deploy. Keep it to a probe, though — do not let a probe failure for a provider you are not currently routing to prevent the service from starting.

Know the failure shape you are looking for. Providers return HTTP 401 with a structured error body for a bad key; the identifiers differ by vendor and are documented in each provider’s errors reference. Match on the status code and the error type field, never on the human message string — those are rewritten without notice and a rotation script keyed on message text will fail open.

Watching the old key go quiet

Revocation should be triggered by an observation, and the observation is that nothing has used the old credential for long enough.

Where the provider reports usage broken down by key, that is the authoritative source and you should use it — it counts calls you have forgotten about, from services you did not know existed, which is exactly the population that breaks. Your own metrics only count the code you instrumented. Emit a counter labelled by credential name on every outbound call as well, so you have both a first-party and a second-party view, and treat disagreement between them as a finding.

“Long enough” is the maximum of a few durations, and it is usually longer than people assume: one full deploy cycle across every service; the longest-running background job, which may have read the key at start and be holding it in memory for hours; the retry horizon of any queue that stores a credential in the job payload, which can be days; and any scheduled job that runs less often than daily. A monthly billing export that reads the key once a month is the classic revocation casualty.

When the counters are at zero across that window, revoke, and do it in business hours with the person who can create a replacement present. Then delete the old secret from the store rather than leaving it — a revoked key still sitting in configuration is indistinguishable from a live one to the next person, and will be tried during an incident.

Where old keys hide

Before revoking, search deliberately. In rough order of how often each is the one that breaks:

• Queued job payloads. Any job that captured a credential at enqueue time carries it until it runs. Jobs should carry a reference to a credential, never the credential itself; if yours do, drain the queue before revoking.
• Long-lived processes holding a value read at start.Resolve credentials per call from a cache with a short TTL rather than once at import time, so a rotation propagates without a restart.
• CI configuration and build artefacts. Test suites that call the real provider, deploy pipelines, and container images with a baked-in environment file.
• Scheduled and infrequent jobs. Anything running weekly or monthly will not appear in a week of usage data.
• Local developer environments and shared notebooks.Not production, but a revocation that breaks the whole team’s local setup on a Monday morning is still an outage of a kind.

One rule makes all of these cheaper next time: never reuse a credential value, and never move one between environments. Rotate forward only, one key per environment per provider, and the search above shrinks to a single well-known set of places. Secrets management when running two providers at once covers the storage and selection pattern that keeps it that way.