The Model Ignores Part of a Long System Prompt

When a model follows nine of your ten instructions, the tenth was not ignored at random. It was too far from the end, contradicted by something else, one of too many simultaneous constraints, or phrased as a prohibition. Those four causes are distinguishable, and each has a different rewrite.

First, confirm it arrived

Before analysing the model’s behaviour, verify the model received what you think it received. Three ways this fails silently:

• The framework moved it. Some abstractions fold the system message into the first user turn, prepend their own text, or append a template you did not write. Print the assembled messages immediately before the call, not the variable you set.
• The model has no system role. Some open-weight chat templates have no system slot, and the content is either merged into the first user message or dropped entirely by the template. Render the chat template and read the result.
• It was truncated. If total input exceeds the window and something in the chain trims, the system message may be what went — especially if the trimming keeps the most recent messages.

import json
print(json.dumps(messages, indent=2)[:4000])
print("system chars:", len(messages[0]["content"]))

Four reasons an instruction gets dropped

1. Position. Attention is not uniform over a long context. Material at the beginning and the end is used more reliably than material in the middle, which is the well-documented lost-in-the-middle effect. An instruction on line 40 of an 80-line system prompt is in the worst position available. Test: move the ignored instruction to the very end of the system message, change nothing else, and re-run twenty times.
2. Conflict. Two instructions that cannot both be satisfied, and the model picks one. These are usually not obvious — “be concise” and “always explain your reasoning” conflict, as do “never speculate” and “always give an answer”. Recency and specificity tend to win, so a later user message beats an earlier system rule. Test: delete the other instructions in the same area and see whether compliance returns.
3. Count. The binding limit is the number of simultaneous constraints, not the token count. A prompt with forty numbered rules will not have forty of them honoured on every response, and adding an emphatic forty-first makes the other forty slightly worse rather than fixing anything. Test: cut the rules to the five that matter and measure compliance on those five. If it jumps, count was the cause.
4. Phrasing. Prohibitions perform worse than instructions. “Do not mention pricing” puts pricing in the context and asks for its absence, which is a harder target than a positive instruction about what to discuss instead. Vague qualifiers — professional, appropriate, concise — are not verifiable, so the model cannot check itself against them.

There is a fifth contributor that is not about the prompt at all: examples beat instructions. If your message history contains turns where the assistant broke the rule — because it was generated before you added the rule, or because a few-shot example predates it — the model reads that as evidence of what happens here. Prune conversation history that demonstrates the behaviour you are trying to stop.

The rewrites, with before and after

Prohibition to instruction

Before: Do not make up product features.
After:  Answer only from the CONTEXT section. If the context does not
        contain the answer, reply exactly: "I do not have that
        information."

The second version is positive, names the source of truth, and specifies the fallback output. It is also checkable in code, which the first is not.

Vague to verifiable

Before: Keep responses concise and professional.
After:  Respond in at most 3 sentences. Do not use bullet points.
        Do not begin with a greeting.

Forty rules to a structure

Before: 40 numbered rules in one block.
After:  ROLE:        one sentence.
        OUTPUT:      the format, exactly.
        RULES:       the 5 that change the answer.
        CONTEXT:     retrieved material, clearly delimited.
        REMINDER:    the 1 or 2 rules most often broken, restated last.

The final reminder is not redundancy for its own sake. It puts the highest-value constraints in the position with the strongest recency, which is precisely the fix for cause 1. Two or three lines, not twenty; a restatement of everything is just the same prompt again.

Buried to positioned

Where a rule must apply to specific retrieved material, put it adjacent to that material rather than in a distant rules block. Instructions immediately before or after the content they govern are followed more reliably than the same instruction fifty lines away — instruction placement covers the pattern.

Move hard constraints out of the prompt

This is the part most pages on this subject omit, and it is the one that actually ends the problem. A prompt is a probabilistic instrument. If a constraint must hold every time — a legal disclaimer, a currency format, an allowed set of categories, a maximum length — enforce it in code and stop asking.

• Enumerations become schemas. “Respond with one of: refund, exchange, escalate” is a rule the model can break. An enum in a structured-output schema is one it cannot — enums versus free text.
• Formats become validators. Validate the output, and on failure either repair it or re-ask with the specific error. Validation and repair covers the loop.
• Length becomes truncation, or a parameter. “At most three sentences” is a suggestion; max_tokens plus a post-check is a guarantee.
• Required text becomes concatenation. A disclaimer that must appear should be appended by your code, not requested from the model. It is shorter, cheaper and certain.

A useful rule of thumb: if a constraint failing would be an incident, it does not belong in a prompt. Deterministic rails and system prompt design develop the division of labour.

What no prompt structure will fix

Some instructions fail because they ask for something the mechanism does not provide, and no rewrite recovers them. Recognising these saves the days that would otherwise go into rephrasing.

• Anything requiring counting. “Exactly 100 words”, “five bullet points”, “three sentences” are approximated, not computed. The model has no running counter and is generating one token at a time. Small counts work reasonably; precise ones do not. Enforce in code.
• Reliable self-assessment. “Only answer if you are confident” asks for a calibrated judgement the model cannot make about itself. Use an explicit abstention path with a checkable condition — the context contains the answer or it does not — rather than an appeal to confidence; abstention covers what does work.
• Consistency across many turns. A rule honoured at turn 2 has to survive a growing context of counter-examples by turn 40. Re-injecting the constraint near the end of each request is the only reliable mitigation, and it costs tokens on every call.
• Instructions about material the model cannot see. “Do not contradict the documentation” is unenforceable unless the documentation is in the context.

One more that is not a limitation but a variable: instruction following differs substantially between models, and between a base model and its instruction-tuned sibling. Before rewriting a prompt for the fifth time, run the existing one against a different model. If compliance jumps, the prompt was not the problem.

Measuring compliance instead of arguing

“It ignores the instruction” is usually an impression formed from three examples. Instruction following is probabilistic, so the real question is the rate, and the rate is what tells you whether a rewrite helped.

import statistics

def compliance(prompt_variant, cases, check, n=20):
    """check(output) -> bool. Returns the rate, not a verdict."""
    hits = 0
    for case in cases:
        for _ in range(n):
            out = call(system=prompt_variant, user=case, temperature=0.7)
            hits += bool(check(out))
    return hits / (len(cases) * n)

for name, variant in VARIANTS.items():
    print(f"{name:24} {compliance(variant, CASES, CHECK):.1%}")

Twenty runs per case, because a single sample cannot distinguish 60% compliance from 95%. Note the deliberate non-zero temperature: at temperature 0 you measure one path through the distribution, and production is not at temperature 0 unless you set it there. Once you have a rate, a rewrite either moves it or it does not, and the argument ends.