Migrating a Prompt's Citation and Source-Attribution Format

Your retrieval-augmented endpoint returns answers with a citations array attached. After the model swap the answers still look cited — the numbered markers are right there in the text — but the array arrives empty on every request, the footnote links render as nothing, and no exception was raised anywhere in the stack.

The symptom: zero citations, no error

Find the parser. It almost always looks like this:

CITE = re.compile(r"\[(\d+)\]")

def extract(answer: str, chunks: list[Chunk]) -> list[Citation]:
    return [
        Citation(marker=m.group(0), chunk=chunks[int(m.group(1)) - 1])
        for m in CITE.finditer(answer)
    ]

It returns an empty list because the new model did not write [1]. It wrote [^1], or (Source 1), or (doc_4b21), or it put the attributions in a table at the bottom instead of inline. The regex does not match, the comprehension yields nothing, and an empty list is a perfectly valid value — so the failure propagates all the way to the user interface as “this answer has no sources” without a single log line. This is the worst class of migration bug: silent, plausible, and invisible to a smoke test that only checks the endpoint returned 200.

Reproduce it in one call. Send the same prompt and the same retrieved chunks to both models, print both raw answers side by side, and look at the markers. You do not need an evaluation harness to see this one.

The three formats a prose parser meets

Across models and across prompt phrasings, attribution lands in three shapes, and a parser written against one of them will not survive contact with the others.

• Inline numeric markers. [1], [1,3], [1][3], or the Markdown footnote form [^1] with a definition block at the end. The multi-source variants are what break parsers that were written when every sentence happened to cite one document.
• Named references. (Source: Q3 revenue memo) or (per the onboarding guide). These carry the document title rather than an index, so index-based lookup fails even when the regex matches, and title matching is fuzzy by nature.
• Trailing reference sections. A “Sources” heading with a list underneath, and no inline markers at all. Every inline-marker parser returns zero here, and the answer is not uncited — it is cited in a place your code never looks.

A prompt that says “cite your sources as [1], [2]” makes one of these more likely, not certain. Instruction-following strength differs between models, and a model that follows the letter of the instruction on short answers can drift to a reference section on long ones where the inline form would be unreadable.

Why the format was never yours to depend on

Formatting habits come from post-training, not from your prompt. Your prompt biases them; it does not define them. That means the citation format is a property of the model you happened to be using, and you built a parser against an undocumented property of a dependency you have no contract with. It worked for a year, which is what made it look like a contract.

There is a second, subtler dependency in the same place. Index-based markers only work if the model numbers its sources in the order you supplied the chunks, and that ordering convention is also a habit rather than a guarantee. A model that groups related sources, or that numbers by order of first mention rather than by input position, will produce markers that parse cleanly and attribute wrongly. That failure is worse than the empty array, because it is invisible in the interface: every sentence has a link, the links go to real documents, and some of them are the wrong document. If your parser resolves an integer to a position in a list, check the resolution against the quoted text before you trust it.

The general treatment of this shape — a model producing text that another program has to consume — is covered in structured output versus function calling and RAG citations. What is specific to a migration is that the failure appears without any code change on your side, and that the parser is usually far from the model call, so nobody thinks to look at it.

Moving citations out of the prose

The durable fix is to stop parsing prose. Ask for the answer and the attributions as separate fields of a schema-constrained response, so the citation channel is validated by the API rather than by your regex.

response = client.messages.create(
    model=MODEL,
    max_tokens=2000,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "answer": {"type": "string"},
                    "citations": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "chunk_id": {"type": "string"},
                                "quote": {"type": "string"},
                            },
                            "required": ["chunk_id", "quote"],
                            "additionalProperties": False,
                        },
                    },
                },
                "required": ["answer", "citations"],
                "additionalProperties": False,
            },
        }
    },
    messages=[{"role": "user", "content": prompt_with_chunks}],
)

Two properties make this survive the next migration. The schema is part of the request, so a model that would have chosen a different prose convention has nowhere to express that choice. And chunk_id is an identifier you minted, so the lookup is exact rather than positional — an off-by-one in the chunk ordering can no longer mis-attribute a sentence to the wrong document.

Anthropic’s API also exposes a first-party citation channel: setting citations to enabled on a document content block makes the response split into text blocks that carry a citations array, each entry naming cited_text, document_index and a location — char_location for plain text or page_location for a PDF. The publisher documents this at platform.claude.com. Where it is available it is strictly better than a schema, because the character offsets are produced by the serving layer rather than copied by the model, so a quote cannot be paraphrased into something that never appeared in the source.

When you cannot move them: a tolerant parser

If the answer text has to stay free-form — because a downstream renderer expects Markdown, say — then make the parser tolerant instead of exact, and make it loud. Match a union of the three families, normalise to a common representation, and treat “zero citations extracted from a non-empty answer over retrieved chunks” as an error rather than as a valid result.

MARKERS = re.compile(
    r"\[\^?(\d+(?:\s*[,;]\s*\d+)*)\]"     # [1] [^1] [1,3]
    r"|\((?:Source|Ref|per)[:\s]+([^)]+)\)",   # (Source: title)
    re.IGNORECASE,
)

def extract(answer, chunks):
    found = _scan(MARKERS, answer, chunks) or _scan_reference_section(answer, chunks)
    if chunks and answer.strip() and not found:
        raise CitationFormatError(
            f"no citation markers found in a {len(answer)} char answer "
            f"over {len(chunks)} chunks; head={answer[:120]!r}"
        )
    return found

The raised error is the point. A tolerant parser that silently returns an empty list on an unrecognised format has reproduced the original bug with more code. Fail, log the first 120 characters of the answer so the new format is in the log line, and add it to the union deliberately — rather than discovering months later that a quarter of your answers shipped uncited.