Managing Context in a Coding Agent

A repository is a few million tokens. A window is a few hundred thousand at best. Coding agents are the workload where the gap between candidate context and available context is largest, which makes them the place every context technique gets tested first.

Why this is the hard case

Four properties combine badly, and each one on its own would be manageable.

• The corpus is enormous and interconnected. Understanding one function may require its callers, its types, the config that drives it and the test that pins its behaviour — spread across five files with no locality.
• The context mutates during the run. Unlike documents, code changes because the agent changes it. A file read at step 3 is stale by step 9, and the stale copy is still in the window claiming to be the file.
• Precision is required. A summary of a function is not a substitute for the function. You cannot compress code the way you compress prose without destroying the thing that made it useful.
• Runs are long. Dozens of steps, each producing tool output. Everything in tool result management applies here at maximum intensity.

The repo map

The first move is to stop trying to put code in the window and put a map there instead: a compressed index of what exists and where, dense enough that the agent can decide what to read. The technique was popularised by the open-source coding assistant Aider, which builds a ranked map of repository symbols and includes it in the prompt so the model can navigate before it reads.

src/billing/invoice.ts
  export class Invoice { id, lines, total(), applyTax(rate) }
  export function renderInvoice(inv: Invoice): string
src/billing/tax.ts
  export function rateFor(region: string): number
  export const REGIONS: string[]
src/api/routes.ts
  POST /invoices  -> createInvoice
  GET  /invoices/:id -> getInvoice

Signatures without bodies. A whole medium repository fits in a few thousand tokens this way, against a few million for the source, and the agent gains the one thing it cannot get from reading files one at a time: knowing what it has not read. An agent without a map explores by guessing filenames, which is both expensive and unreliable.

Two refinements are worth the effort. Rank the map by relevance to the current task — files mentioned in the request, files recently touched, files that import the ones in play — and include more detail for the top-ranked entries. And keep the map in the stable tier of the prompt so it stays cacheable; regenerating it every step for marginal freshness gains is a false economy.

File windows, not files

The second move is to stop reading whole files. Most reads need a function, not a module, and a 2,000-line file read to see one method has spent 90% of its tokens on material the agent will never reference.

• Read by symbol where possible. read_symbol(“Invoice.applyTax”) beats read_file(“invoice.ts”) on every axis, and an AST or tag index makes it cheap.
• Read ranges with context. When a line number is known — from a stack trace or a search hit — read the surrounding forty lines rather than the file.
• Prefer search over browse. A grep returning ten matching lines with their locations costs a fraction of the three files the agent would otherwise open to find them.
• Deduplicate reads. Agents re-read the same file constantly. A read cache keyed on path and content hash turns the second read into a one-line reference to the first, and this alone reclaims a surprising share of a long run’s window.

The eviction policy

This is where a coding agent lives or dies, and it is the part most implementations leave to “drop the oldest.” A policy with a stated priority order, pins and a score:

// NEVER evicted, regardless of age or score.
const PINNED = new Set([
  "task",            // the original request
  "plan",            // the current plan
  "repo_map",        // navigation
  ...openFiles,      // files with uncommitted edits by this agent
  ...failingTests,   // the errors currently being worked on
]);

function evictionScore(item) {
  const age   = step - item.step;              // steps since it entered
  let score = 100;
  score -= age * 4;                            // recency
  score += item.referencedSince ? 40 : 0;      // mentioned after arriving
  score += item.kind === "error" ? 30 : 0;     // errors outlive successes
  score -= item.stale ? 60 : 0;                // superseded by a later edit
  score -= Math.log2(item.tokens) * 3;         // large items cost more to keep
  return score;
}

function evict(items, need) {
  const cands = items.filter(i => !PINNED.has(i.id))
                     .sort((a, b) => evictionScore(a) - evictionScore(b));
  let freed = 0;
  for (const c of cands) {
    if (freed >= need) break;
    freed += collapse(c);        // replace with a one-line summary + handle
  }
}

Four decisions inside that are worth defending. Stale beats old: a file read before the agent edited it is actively harmful, not merely useless, and the -60 makes it the first thing to go — this is the single most valuable term in the function. Errors outlive successes, because the failing test is why the agent is doing what it is doing. Collapse rather than delete: the item becomes a one-line note plus a handle, so the agent knows the read happened and can redo it instead of rediscovering the need from scratch. And pins are a set, not a score, because there is no weighting under which evicting the current plan is correct.

The agent’s own output is context too

The block that grows fastest in a long coding run is often not file reads at all. It is the agent’s own emitted code — every diff, every full-file rewrite, every explanation, all of it billed at the output rate on the way out and then at the input rate on every subsequent turn.

Three consequences follow. Prefer diffs to whole-file rewrites, which reduces output tokens by an order of magnitude on a large file and leaves a much smaller artefact in the transcript. Once an edit is applied, collapse the diff to a one-line note — the file system is now the source of truth, and keeping the diff means the window holds two descriptions of the same state. And write plans and findings to files rather than into the transcript, where they survive compaction and can be edited in place instead of restated.

The staleness problem deserves one more line because it is the failure most specific to this workload. Every file read is a snapshot with a timestamp, and the agent’s own edits invalidate snapshots silently. Track a content hash per read and mark an item stale the moment a write touches that path — that single bookkeeping step is what makes the -60 term above computable, and without it the eviction policy has no way to distinguish a useful old read from an actively misleading one.