Storing Prompts as Files Instead of Strings in Code, for Better Diffs

The argument for taking prompts out of source code is not tidiness. It is that a prompt’s wording is the behaviour, and a diff that shows a modified string literal on one long line hides the only part of the change anyone needs to read.

What a string literal does to a diff

Consider a system prompt held in a TypeScript template literal. Someone changes “summarise” to “summarise in at most three sentences” and reflows the paragraph. Because the text is one logical line inside a literal, the diff shows a red line and a green line, both several hundred characters long, and the reviewer’s eye has nowhere to land. Reflowing anything makes it worse: every subsequent line moves, so an eight-word edit renders as a wholesale replacement of the block.

There are secondary effects that are worse than the visual one:

• Escaping corrupts the reading. Quotes, backslashes and braces in the prompt have to be escaped for the host language, so the text a reviewer reads is not the text the model receives. JSON examples inside prompts are the common casualty.
• Indentation leaks in. A prompt indented to match surrounding code carries that indentation to the model unless it is stripped, and whether it is stripped depends on which dedent helper somebody used.
• Blame is useless. Line-based blame on a multi-hundred-character line tells you who last touched the whole prompt, not who wrote the clause you are asking about.
• Non-engineers cannot contribute. The person who most wants to fix the wording — support lead, domain expert — is not going to edit a Python file to do it.

A layout that survives a year

One prompt per file, one directory, one naming convention, and metadata in front matter so the file is self-describing:

prompts/
  support/
    triage.md            # the prompt text, with front matter
    triage.schema.json   # the output schema this prompt must satisfy
  billing/
    explain-invoice.md
  README.md              # who owns what, and how to change one

---
id: support.triage
version: 7
owner: support-eng
model_hint: mid-tier
variables: [customer_message, account_tier, locale]
---
You are triaging inbound support messages.

Classify the message into exactly one category from this list:
billing, technical, account, other.

Return JSON matching triage.schema.json. Return nothing else.

Account tier: {{ account_tier }}
Locale: {{ locale }}

Message:
{{ customer_message }}

Markdown with YAML front matter is a reasonable default because every editor renders it, every review tool shows it word-wrapped, and the front matter gives you a place for the fields you would otherwise scatter across the codebase. The important properties are that the text is unescaped, that it is line-broken at sentence or clause boundaries, and that the file is the only place that wording exists.

Break lines deliberately. A prompt written as one paragraph per line diffs at paragraph granularity; a prompt written one sentence per line diffs at sentence granularity, which is what you want in review. It costs nothing at runtime as long as your loader does not collapse newlines, and it makes a one-clause edit render as a one-line change.

Loading them without a runtime surprise

Externalising the text introduces a failure mode that a string literal did not have: the file can be missing at runtime. A missing prompt is a production outage that no local run reproduced, and it happens for boring packaging reasons — the files were not included in the wheel, the Docker image copied only the source directory, the serverless bundler tree-shook a directory nothing statically referenced.

Three defences, in order of preference. First, load and validate every prompt at startup rather than on first use, so a missing or malformed file fails the health check instead of the first customer request. Second, add a test that enumerates the directory and asserts that every file parses, that its declared variables match the placeholders in its body, and that every prompt id referenced in code resolves. Third, if your deployment format makes file paths fragile, embed the files at build time — but embed them from the same source of truth, never by copying the text back into code.

# prompts/loader.py
from pathlib import Path
import re, yaml

ROOT = Path(__file__).parent
_CACHE: dict[str, dict] = {}

def _parse(path: Path) -> dict:
    raw = path.read_text(encoding="utf-8")
    if not raw.startswith("---"):
        raise ValueError(f"{path}: missing front matter")
    _, front, body = raw.split("---", 2)
    meta = yaml.safe_load(front)
    declared = set(meta.get("variables") or [])
    used = set(re.findall(r"\{\{\s*(\w+)\s*\}\}", body))
    if declared != used:
        raise ValueError(f"{path}: declared {declared} but used {used}")
    return {"meta": meta, "body": body.strip()}

def load_all() -> dict[str, dict]:
    """Call once at startup. Raises rather than serving a missing prompt."""
    for path in ROOT.rglob("*.md"):
        parsed = _parse(path)
        _CACHE[parsed["meta"]["id"]] = parsed
    return _CACHE

def get(prompt_id: str) -> dict:
    return _CACHE[prompt_id]

The declared-versus-used check in there is the highest-value five lines in the file. A renamed variable that the template still references produces an empty substitution, and an empty substitution is a prompt that reads fine and behaves badly — the sort of defect that survives to production because nothing raised.

Making the diff read like prose

Once the text is in files, two Git settings make review markedly better. Word-level diffing shows which words changed inside a line rather than replacing the line, which is exactly the granularity a prompt edit lives at:

# For one review, on the command line:
git diff --word-diff=color -- prompts/

# Permanently, for prompt files only, via .gitattributes:
#   prompts/**/*.md diff=prompt
# and in .gitconfig:
#   [diff "prompt"]
#     wordRegex = "[A-Za-z0-9_]+|[^[:space:]]"

The second is to make sure these files are not marked binary or excluded from review by a generated-file rule, which happens by accident when someone adds a broad pattern to .gitattributes for build output. If a prompt file ever shows as “binary files differ”, that is usually a stray byte-order mark or a CRLF setting, both of which are worth fixing at the source since they also reach the model.

What externalising actually costs you

It is not free, and pretending otherwise is how teams end up with a half-migrated codebase where the important prompts are still in literals.

• You lose the compiler. A typo in a prompt id is now a runtime KeyError rather than a build error. Generating a typed constant per prompt from the directory listing gets it back, and is worth doing once the count passes about a dozen.
• You gain a whitespace surface. Trailing spaces, final newlines and indentation are now content. Normalise on load, and put a linter rule on the directory, because a trailing space before a closing delimiter has changed model behaviour often enough to be worth ruling out.
• Two sources of truth become possible. If prompts are also editable in a hosted tool, decide which one wins and make the other read-only. A prompt file that production ignores is worse than no file, because reviewers will keep reviewing it.
• Tests must load real files. A test that inlines the prompt text it expects has re-created the original problem inside the test suite. Assert on properties of the loaded file — that it declares the right variables, that it mentions the required output format — rather than on its full text, which would make every wording change a test change.

None of that is a reason to keep prompts in literals; it is a reason to do the move once, properly, rather than for the two prompts somebody happened to be editing. The test that makes the migration finishable is a lint rule: fail the build on any string literal in the source tree longer than a few hundred characters that contains a newline and a second-person instruction. It will produce false positives, and the exemptions you add to it are a useful inventory of what is left.