Guardrails: Input Filtering vs Output Filtering
4 min read · updated August 3, 2026
Guardrails are usually discussed as one thing. They are two, they sit on opposite sides of the expensive call, and that placement decides both what they can catch and what they cost you.
Two places, different economics
An input check runs before inference. If it blocks, you have spent a few milliseconds and no inference tokens — and for a cheap rule, its cost rounds to zero. An output check runs after inference. By the time it blocks, the tokens are generated and billed, and if you were streaming, some of them are on the user’s screen.
That asymmetry is the single most useful fact in this comparison. It means input checks should be maximally cheap and permissive-by-default, existing to remove the obvious, while output checks are where you spend real budget because they are the last thing before harm.
One structural point applies to both sides. If the guardrail is itself a language model, it reads the same untrusted text the main model does, so it inherits the same vulnerability — content can address the judge as easily as it addresses the assistant. That does not make model-based guardrails useless; it means they should be given the narrowest possible question (“does this text contain a credential?” beats “is this safe?”), should never be handed tools, and should not be the only thing standing in front of an irreversible action. Deterministic checks have no such problem, which is why they belong at the bottom of both stacks.
What input filtering can and cannot reach
Reaches: requests that are obviously outside your product’s scope; known-bad patterns; oversized or malformed input; prompts containing secrets your users should not be pasting; the low-effort share of injection and jailbreak attempts; and the structural rules that are not judgements at all — length limits, allowed languages, allowed content types.
Cannot reach: anything that depends on what the model will do. A benign-looking question can produce a harmful answer, and a question that reads alarming can have a perfectly ordinary answer. It also cannot decide the injection question, for the reason the whole cluster keeps returning to — instruction-ness is not a property of the text.
The characteristic failure of input filtering is false positives on exactly the users you most want: security teams, medical professionals, researchers and translators all write inputs that a keyword-oriented filter dislikes.
What output filtering can and cannot reach
Reaches: the harm itself, which is a much smaller and more concrete target than the set of ways someone might ask for it. Concretely — secrets and personal data in the response, disallowed content regardless of how it was elicited, hallucinated citations checked against the retrieved sources, unexpected outbound URLs, and tool calls inconsistent with the user’s request. One output check covers every jailbreak family at once, which no input check can claim.
Cannot reach: the cost already incurred, and any side effect that has already happened. If the model called a tool before the check ran, filtering the text afterwards is theatre. This is why tool authorisation must be its own gate rather than a post-generation filter.
Streaming is the operational complication. You either buffer, losing the perceived-latency benefit that streaming existed for, or you check incrementally and accept that a violating token can reach the screen before you retract it. A middle path that works well in practice: stream, run cheap deterministic checks per chunk, and hold the final segment until a full-response check completes.
The comparison
| Property | Description |
|---|---|
| cost when it fires | Input: near zero -- no inference happened. Output: full inference cost, already billed, plus the second check's own cost. |
| latency added | Input: adds to time-to-first-token, so it is the more visible one for a cheap rule. Output: adds to completion, and forces buffering if the check needs the whole response. |
| coverage | Input: the ways a request can be phrased -- an unbounded set. Output: the ways harm can appear -- a bounded, enumerable one. |
| false positive cost | Input: user is refused before anything happens, which is visible and annoying. Output: user watched an answer appear and vanish, which is worse. |
| attacker's view | Input: probeable, and each probe is free. Output: probeable, and each probe costs the attacker an inference too. |
| best for | Input: scope, size, structure, known-bad. Output: data leakage, policy, grounding, URLs, tool-call consistency. |
Building both without doubling latency
The design that holds up is a cheap deterministic layer on both sides and an expensive judgement layer on one. Practically:
- Input, always, deterministic only. Length caps, content type, language, rate limits, regex for credentials. Sub-millisecond, no model call.
- Input, conditionally. A small classifier for injection or policy, run in parallel with the start of the main call rather than in front of it — cancel the generation if the classifier objects. You pay a little inference on blocked requests and no added latency on the overwhelming majority that pass.
- Output, always, deterministic. Secret patterns, URL host allowlist, personal-data patterns, forbidden strings. Cheap enough to run per chunk while streaming.
- Output, where the harm justifies it. A model-based check for policy or grounding, on the full response. Reserve it for surfaces where a bad answer is expensive — anything public, anything regulated, anything that triggers an action.
- Tools, always, enforced. Not a guardrail at all: an authorisation gate that runs before the call. Neither filter substitutes for it.
If you can only build one side, build output. It is closer to the harm, it covers attack families you have not enumerated, and it fails in the direction of blocking damage rather than blocking users. The tuning method — thresholds, precision and recall on your own data — is in building a content safety layer.