Redacting API Keys From Recorded LLM Cassettes

You opened a test fixture and found a line reading authorization: Bearer sk-... with the rest of the key intact, committed six months ago. The instinct is to edit the file and push. That is the third step, not the first, and doing it first is what turns a contained incident into an uncontained one.

You just found this

The line looks like one of these, depending on the tool and provider:

# VCR.py cassette (YAML)
  request:
    headers:
      authorization: ['Bearer sk-proj-REDACTED-BUT-IT-WAS-REAL']
      x-api-key: ['sk-ant-REDACTED-BUT-IT-WAS-REAL']

# nock fixture (JSON)
  "reqheaders": { "authorization": "Bearer sk-proj-..." }

# a query-string style key, which header filters miss entirely
  uri: https://generativelanguage.googleapis.com/v1beta/models/...?key=AIza...

Recording tools capture what went over the wire, and your credential went over the wire. Every recording library in this cluster does this by default; none of them redact unless told to.

Rotate first, clean second

Treat the key as public from the moment it was committed. It is present in every clone anyone ever made, in every fork, in the CI cache, in any mirror, and in whatever scrapes public repositories — and if the repository is public, assume it was scraped within minutes of the push rather than eventually.

1. Revoke the key in the provider’s console. Revoke, not “create a new one and leave the old one” — an unrevoked key is a live key regardless of what your application now uses.
2. Issue a replacement and put it wherever secrets belong for this project. If the leaked key was a long-lived personal one, this is the moment to make the replacement a scoped, per-environment key instead.
3. Check the provider’s usage log for the exposure window before you close the incident. That is the only evidence about whether the key was used, and it is the thing you will be asked for.
4. Only now start editing files. Rotation makes everything below cleanup rather than containment, which is a much calmer job.

Stop recording it

Redaction is configured at record time in every one of these tools. A cassette recorded before you added the filter still contains the secret — the filter is not applied on replay, and it will not retroactively clean anything.

VCR.py filters headers, query parameters and form fields, and takes a callback for anything in a body:

import vcr

llm_vcr = vcr.VCR(
    filter_headers=[
        ("authorization", "REDACTED"),
        ("x-api-key", "REDACTED"),
        "openai-organization",
        "set-cookie",
    ],
    filter_query_parameters=[("key", "REDACTED")],
    filter_post_data_parameters=["api_key"],
    before_record_response=scrub_response,
)

The tuple form replaces the value with a placeholder; the bare string form removes the header entirely. Prefer the tuple: a cassette that still has an authorization header with a fake value replays more faithfully, and a reviewer can see that the header was sent at all. VCR.py’s advanced usage page documents both forms, plus before_record_request and before_record_response for anything a header filter cannot reach.

For the other tools in this cluster: nock’s recorder captures request headers only when you enable it, so the simplest defence is to leave that off and strip the fixture in a post-processing step you own. WebMock does not record at all, so the exposure there is not the fixture but the error dump — an unstubbed request prints its headers into the CI log, which is a log-redaction problem rather than a fixture one, and is the same class of mistake covered in what to log.

Three things header filtering does not catch, and all three have bitten people:

• Keys in query strings. Some providers take the credential as a URL parameter, and the URL is stored as the cassette key. Filter query parameters explicitly.
• Secrets in the request body. A prompt containing a customer record, a token, or an internal URL is now a committed file. Filtering runs on headers, not on your prompt.
• Identifiers in the response. Organisation ids, project ids and account emails come back in headers and error messages. Not a credential, but not something to publish either.

Clean what is already committed

Deleting the line in a new commit removes it from the working tree and from nothing else. It is still in the object database and still reachable from every prior commit, so a clone still has it and the GitHub UI will still show it on the old commit.

Rewriting history with a tool built for it — git filter-repo is the currently recommended one — removes it from the commits, but that rewrite changes every commit hash after the touched one, which means a force push and every collaborator re-cloning. Pull requests, forks and cached views on the hosting provider can retain the old objects regardless.

Which is the real argument for the ordering in this page: history rewriting is disruptive, incomplete and slow, and it is only worth doing after rotation has already made the secret worthless. If you have rotated, a plain deletion in a normal commit is often an acceptable end state, and you can spend the effort on the check below instead.

A check that fails the build

Configuration you added once will be missed by the next person who adds a fixture directory, or by the same tool run in a different mode. Something automated has to look.

1. Add a secret scanner as a pre-commit hook. gitleaks and detect-secrets both ship provider key patterns and both run in a couple of seconds on a normal diff.
2. Add the same scan to CI, so a commit made without hooks installed still fails. A local-only hook is a suggestion.
3. Add one plain test that walks your fixture directory and asserts no file contains a live-looking key prefix. It is a handful of lines, it fails with a filename, and it survives the day someone changes recording library.
4. Enable the hosting provider’s push protection if you have it. It is the only layer that acts before the object exists on a server, and it catches the case the other four miss: a commit pushed from a machine that never ran your hooks, on a branch that never reached CI.
5. Record with a deliberately fake key. Point the recording run at a gateway or a proxy with a throwaway credential, so even a missed filter captures a value that was never worth anything.