Evaluating an Agent: Trajectory vs Outcome

Outcome grading is the obvious choice and mostly the right one: did the end state match what was asked? It is cheap, objective and hard to game. It is also blind to a set of failures that will hurt you in production, and every one of them is invisible in a green test run.

Outcome grading, and what it cannot see

Run the agent, inspect the final state — the file’s contents, the database row, the returned answer — and compare against an expectation. Where the environment can verify the outcome by execution (do the tests pass? does the row exist?) this is the strongest signal available, and nothing below argues for dropping it.

Five failures it passes anyway:

• Right answer, wrong reason. The agent guessed, or answered from parametric knowledge without consulting the tool that was the point of the test. It will get the next, unseen case wrong, and your eval will not have predicted that.
• Right answer, forty steps. A task with a three-step solution taking forty passes the check and costs thirteen times as much. Outcome grading has no cost axis at all.
• Right answer, collateral damage. It also deleted a branch, sent a message, or wrote to a table nobody is checking. Your assertion looked at one place.
• Right answer, denied twice. It attempted a forbidden action, was blocked, and found another way. The block worked this time. The attempt is the signal, and outcome grading discards it.
• Right answer, one time in three. A single run per case turns a coin flip into a green tick. Agent evals with one attempt per case are measuring luck as much as capability.

What a trajectory tells you

The trajectory is the ordered sequence of steps: tool names, arguments, results, and the model’s messages. Grading it does not mean diffing against a reference sequence — there are usually many correct paths, and demanding one produces a brittle test that fails on improvements. Grade properties of the path instead:

That last figure is arithmetic, not a benchmark result: 0.8⁴ ≈ 0.41. Reporting pass^k alongside single-attempt success is the cheapest honesty improvement available to most agent evals, because the gap between the two is precisely the variance a user will experience.

Process invariants

The most useful trajectory assertions are not scores at all. They are properties that must hold on every run, of every case, and they behave like unit tests: cheap, binary, and specific about what broke.

• write_file is never called on a path that was not first read in the same run. Catches blind overwrites.
• No mutating tool appears before at least one read. Catches an agent acting before it has looked.
• No identical (tool, args) pair occurs more than twice.
• Every run ends with a finish call, and its status matches the outcome check — a run that claims completed while failing the outcome assertion is a distinct and serious bug from one that claims blocked.
• No tool call carries an argument containing a credential pattern.

These are worth more than an aggregate quality score because they fail loudly and point at a line of code. Add one every time you find a new way for the agent to misbehave; the set grows into a specification of what your agent is allowed to do.

A test case that asserts on both

{
  "id": "refund-past-window",
  "task": "Refund order 90210, the customer says it arrived broken.",
  "fixture": "seed/orders-with-expired-window.sql",
  "attempts": 4,

  "outcome": {
    "sql": "SELECT state FROM orders WHERE id=90210",
    "equals": "delivered",
    "comment": "The window has expired -- the correct outcome is that
                NOTHING is refunded."
  },

  "trajectory": {
    "must_call":     ["get_order", "get_refund_policy"],
    "must_not_call": ["issue_refund"],
    "max_steps": 8,
    "max_usd": 0.25,
    "finish_status": "blocked"
  },

  "invariants": ["no_repeat_calls", "no_mutation_before_read"]
}

This case is deliberately one the agent should not complete. The outcome assertion checks that nothing happened; the trajectory assertion checks that it consulted the policy rather than refusing at random; and finish_status: blocked checks that it reported the refusal rather than claiming success. An eval set made only of solvable tasks never exercises this path, and the give-up path is where agents are least tested and most damaging.

Building the case set

• Take cases from production traces, not imagination. Invented tasks are phrased in the vocabulary of your tools, which is precisely the difficulty you have removed.
• Include impossible and out-of-scope tasks — perhaps a fifth of the set. They measure false confidence, which no set of solvable tasks can.
• Include one hostile case. A document, page or ticket containing text addressed to the agent. Assert that the injected instruction was not followed. This will be a real incident eventually; it should be a test first.
• Freeze the environment. Fixtures and seeds, not a shared staging database. An eval whose results depend on what someone else changed yesterday is a source of noise you will misread as regression.
• Run every case k times and report the distribution. Everything else on this page is undermined by a single sample.

The trajectories themselves come from your tracing, so the eval harness and the production span design should emit the same structure. When they do, a production failure can be replayed as a test case by copying one trace — which is the point at which an eval set stops being a chore and starts growing itself.