Structured Context: XML, JSON or Prose?

The same content marked up three ways is three different token counts and three different levels of boundary clarity. The token half is arithmetic you can do; the accuracy half is task-specific and the honest answer is a harness rather than a table.

What the encoding is for

Marking up context does exactly two jobs. It tells the model where one block ends and the next begins, and it labels what each block is. That is all. Anything an encoding does beyond those two things is overhead you are paying for.

The two jobs matter most where confusion is most likely: separating retrieved documents from each other, separating instructions from data the instructions operate on, and separating user-supplied text from everything else. That last one is not only a clarity concern — an unmarked boundary between your instructions and text a user pasted is the structural condition prompt injection exploits, and delimiters are a partial mitigation rather than a cosmetic choice.

Note that this page is about the container, not the contents. How to word the instruction inside the tag belongs to prompt structuring; what is at stake here is the per-block cost of the container and what it does to boundaries.

Where the tokens go

Structural overhead is countable from the syntax itself, without running anything. Take one record with five fields and a 200-token text body, repeated forty times.

The escaping term is the one people forget, and it is the reason JSON is usually the most expensive encoding for text-heavy payloads. A body containing quotes, newlines and backslashes must have every one of them escaped, and \n as two characters where a real newline was one is a token cost repeated on every line break. A 200-token paragraph with twelve line breaks pays for those twelve breaks twice over — once as escape sequences that tokenize poorly, and again because escaped text disrupts the tokenizer’s ordinary merges. For structured data with short values, JSON is fine; for documents, it is the wrong container.

There is also a documented practice worth citing rather than re-deriving: Anthropic’s own prompting documentation recommends XML-style tags for delineating parts of a prompt to Claude models. That is vendor guidance about one model family, not a general benchmark result, and it should be weighted as such — but it is a real, checkable recommendation from the people who trained the models, and it is why tag-style delimiters are the common default in Claude-targeted prompts.

Counting it on your own payload

The overhead depends on your field names, your value lengths and your tokenizer, so the only number worth having is the one you measure. It takes a few lines.

const records = loadYourRealRecords();   // 40 of them, not a toy

const encoders = {
  prose: rs => rs.map(r =>
    r.title + " (" + r.source + ", " + r.date + ")\n" + r.body
  ).join("\n\n"),

  markdown: rs => rs.map(r =>
    "## " + r.title + "\n_" + r.source + " · " + r.date + "_\n\n" + r.body
  ).join("\n\n---\n\n"),

  tags: rs => rs.map(r =>
    "<doc source=\"" + r.source + "\" date=\"" + r.date + "\">\n" +
    r.body + "\n</doc>"
  ).join("\n"),

  json: rs => JSON.stringify(rs, null, 2),
};

const base = count(records.map(r => r.body).join(""));   // content only
for (const [name, enc] of Object.entries(encoders)) {
  const total = count(enc(records));
  console.log(name, total, "overhead:", total - base,
              ((total - base) / base * 100).toFixed(1) + "%");
}

Run it against the tokenizer for the model you actually call — tokenizers differ between families and punctuation is exactly where they differ most, so a count from the wrong one can be misleading in both directions. The mechanics of client-side counting are the tokens cluster’s subject.

Do the accuracy half yourself too, with the same harness shape as the placement experiment: same content, same instruction, three encodings, mechanical grading, enough cases to see a small effect. Anyone offering you a general answer to “which format is most accurate” is over-generalising from one model and one task.

Choosing per block

There is no reason to use one encoding for the whole context, and good reasons not to. Match the container to the block.

• Instructions: prose. Written for a reader, and models were trained on enormous quantities of instructional prose. Wrapping a paragraph of guidance in JSON adds cost and helps nothing.
• Retrieved documents: tags at block level. One opening tag with attributes for provenance, the body verbatim inside, one closing tag. Unambiguous boundaries, provenance available for citation, and the overhead is a fixed few tokens per document rather than per field.
• Records and tables: whichever is denser. For many short fields, a header row and delimited lines beat both JSON and per-field tags by a wide margin — the field names are paid once for the whole table instead of once per row.
• Tool results: whatever the tool emits, bounded. Do not re-encode a JSON API response into tags; you pay to transform it and gain nothing. Projecting away unused fields saves far more than any container choice.
• User-supplied text: always delimited, always the same way. Consistency matters more than which delimiter. And escape or strip anything in the user’s text that looks like your delimiter, or the boundary you are relying on is one the user can forge.

The nesting trap

The most expensive format mistake is not choosing the wrong encoding; it is applying a good encoding at the wrong granularity. Tags around each of eight fields in each of forty records is 640 tags. The same information with one tag per record and the fields as plain lines inside is 80. The boundary information gained by the deeper nesting is approximately zero, because nothing was ambiguous at field level in the first place.

The rule that falls out: structure at the level where ambiguity actually exists, and no deeper. Two documents next to each other are ambiguous — tag them. A title followed by a date on the next line is not ambiguous — do not.

One genuine argument for heavier structure is worth keeping in view: when you intend to reference a specific part later (“in document 3, section 2”), explicit identifiers earn their tokens because they make the reference resolvable and make a context dump searchable. Structure for addressability, not for tidiness.