A Rules Engine Next to a Model

Some decisions must give the same answer every time, must be explicable by naming the rule that produced them, and must change only when somebody deliberately changes them. A language model provides none of those three properties. This is not a quality problem that a better model fixes; it is a category difference, and the design that follows from it is to let the model handle language and let a rules engine handle the decision.

Where the line goes

A decision belongs in rules if any of the following is true of it. Most of the decisions people are tempted to hand to a model satisfy two or three:

• It is written down somewhere already — in a policy, a contract, a regulation, a price list. A rule that exists in prose has a canonical form, and reproducing it probabilistically is strictly worse than encoding it.
• Two identical inputs must give one answer. Two customers with the same facts receiving different refunds is a fairness problem before it is an engineering one, and it is the single most common complaint against automated decisions.
• Somebody may have to justify it later. If the answer to “why was this declined” must name a criterion, the decision needs a rule that fired, not a token distribution.
• Changing it requires approval. A rule can be diffed, reviewed, tested and dated. A prompt edit that shifts behaviour on some inputs and not others cannot be reviewed in the same sense.
• The cost of being wrong is asymmetric and large. Eligibility, entitlements, tax treatment, credit limits, dosing, safety interlocks, access control.

Three things a rules engine gives you

The third is the one that decides the argument in regulated environments, and it is worth stating precisely: with a rule set you can answer “which of last year’s decisions would this change affect?” by replaying them. With a prompt change you can sample.

The routing pattern

1. The model reads. It turns an email, a call transcript or a form into a structured record: what happened, when, what the customer is asking for. This is structured extraction and models are extremely good at it.
2. Facts are verified, not accepted. Every extracted field that a rule depends on is checked against a system of record — order date from the orders table, tier from the subscription table. An extracted fact that cannot be verified stops the automated path.
3. The rules decide. Deterministically, on verified fields only, producing an outcome and the id of the rule that produced it.
4. The model writes. It turns the outcome and the rule into a sentence a person will understand, constrained to state the decision it was given.
5. Everything is logged — the extracted record, the verification results, the rule trace, the generated text. The decision is reproducible from the record without the model.

Step 2 is where most implementations of this pattern are weak. An extraction fed straight into a deterministic rule engine produces a deterministic decision on possibly-wrong facts, which is a confident wrong answer with an audit trail. The verification step is described in checking model output against a database.

A decision table, and its evaluator

Most business rules are a decision table: a list of conditions, in priority order, each with an outcome. Written as data, it can be read by the person who owns the policy — which is the point, because they are the one who will change it.

# refund_policy.yaml — version 12, effective 2026-07-01
rules:
  - id: R-01
    when: {days_since_delivery: {gt: 90}}
    then: {outcome: decline, reason: "outside the 90-day return window"}

  - id: R-02
    when: {item_condition: used, item_category: hygiene}
    then: {outcome: decline, reason: "hygiene items cannot be returned once used"}

  - id: R-03
    when: {fault: manufacturing, days_since_delivery: {lte: 365}}
    then: {outcome: full_refund, reason: "manufacturing fault within 12 months"}

  - id: R-04
    when: {days_since_delivery: {lte: 30}, item_condition: unopened}
    then: {outcome: full_refund, reason: "unopened within 30 days"}

  - id: R-05
    when: {days_since_delivery: {lte: 30}, item_condition: opened}
    then: {outcome: partial_refund, rate: 0.80, reason: "opened within 30 days"}

  - id: R-06
    when: {order_value_minor: {gt: 200000}}
    then: {outcome: escalate, reason: "over EUR 2,000 requires human approval"}

  - id: R-99
    when: {}
    then: {outcome: escalate, reason: "no rule matched"}

OPS = {
    "gt":  lambda a, b: a >  b,
    "gte": lambda a, b: a >= b,
    "lt":  lambda a, b: a <  b,
    "lte": lambda a, b: a <= b,
    "in":  lambda a, b: a in b,
}

def matches(condition, facts) -> bool:
    for field, expected in condition.items():
        if field not in facts:
            return False                      # unknown fact never matches
        actual = facts[field]
        if isinstance(expected, dict):
            for op, operand in expected.items():
                if not OPS[op](actual, operand):
                    return False
        elif actual != expected:
            return False
    return True


def decide(rules, facts) -> dict:
    trace = []
    for rule in rules:
        hit = matches(rule["when"], facts)
        trace.append({"rule": rule["id"], "matched": hit})
        if hit:
            return {"decision": rule["then"], "fired": rule["id"],
                    "trace": trace, "policy_version": 12}
    raise AssertionError("rule set has no catch-all; add R-99")

Two design choices in there are deliberate. First-match-wins with an explicit order makes the policy readable top to bottom and makes conflicts a question of ordering rather than of resolution strategy. And an unknown fact never matches, so a missing field routes to the catch-all escalation rather than silently taking a branch — which is the behaviour you want when extraction failed.

Two further disciplines make this survive: every rule change is a new version with an effective date, and the rule set is replayed against a library of past cases before it ships, so the diff is expressed as “these 41 historical decisions would change” rather than as a YAML patch. Both are trivial with a table and impossible with a prompt. DMN is the standard notation for this if you need something a business analyst can edit in a tool rather than in a text file.

The arrangement that looks equivalent and is not

The tempting inversion is to let the model decide and have the rules check afterwards, rejecting decisions that violate policy. It sounds like the same guarantee arrived at from the other side. It is not, for three reasons.

• A checker only catches what it can express. If the check could express the full policy, it could have made the decision; it is only cheaper as a checker if it is weaker, and the gap is exactly the set of violations that get through.
• The failure mode is a rejection loop. When the check fails, something has to happen. Re-prompting until the model produces an acceptable answer optimises for passing the check rather than for being right, and the number of attempts becomes a hidden variable in the decision.
• The audit trail says the wrong thing. “A model proposed this and a check did not object” is a materially weaker statement than “rule R-03 fired”, and the difference is exactly what an auditor is looking for.

Post-hoc checking is a good safety net on top of a rules-first design. It is not a substitute for one, and the same argument appears in putting deterministic rails around a model.

Keeping the rules honest

• One owner per rule set, named, who is not an engineer. Policy that lives only in a repository drifts from the policy the business believes it has.
• Every escalation is a signal. A rule that escalates forty per cent of cases is either badly written or the policy genuinely has a gap; both are worth knowing and only the escalation rate reveals it.
• Test the table, not the wrapper. A case library of input records with expected outcomes, run in CI, is what stops a reordering from silently changing last year’s answers.
• Watch for the rule set becoming a program. Once rules start referring to the results of other rules and ordering matters in subtle ways, you have written code in YAML. That is the point at which either the logic moves into code with tests, or the problem is actually a constraint satisfaction problem and wants a solver.