Context Budget Allocator
Splits a context window across the system prompt, tool schemas, retrieval, history and the current message, and says how many turns still fit.
History can grow to 260 turns before anything else has to give.
- Input budget — window minus the answer
- 124,000
- System prompt (0.6%)
- 700
- Tool schemas (1.5%)
- 1,800
- Retrieval — 8 × 512 (3.3%)
- 4,096
- History — 12 turns (4.4%)
- 5,400
- Current message (0.2%)
- 250
- Total sent
- 12,246
- Free
- 111,754
- History turns that fit
- 260
- Input cost of one call at that size
- $0.04
A context window is a budget, and the fixed costs come first
Everything sent to a model competes for one number, and the parts do not compete equally. The system prompt and the tool schemas are fixed: paid on every call, identical every time, and invisible in most logging. Retrieval is semi-fixed — you chose top-k once and it applies to every request. History grows without anyone deciding it should. The user's actual question, the only part they wrote, is usually the smallest line in the table above.
Laying it out this way makes two things obvious. First, where the window really goes: a generous tool belt and eight retrieved chunks can take more of the budget than the entire conversation. Second, what to cut when you overflow, which is a question people answer badly under pressure by truncating the oldest history — the cheapest fix in tokens is often to retrieve fewer chunks, since each one costs as much as a whole turn.
Fitting is not the same as working. Retrieval accuracy degrades over long contexts, and material in the middle of a large prompt is attended to less reliably than material at either end, so a request that fits with room to spare can still answer worse than a shorter one. Treat the free-tokens figure as a hard ceiling rather than a target, and if you are anywhere near it, the fix is compressing what you send rather than buying a bigger window.