Agent Evaluation: Trajectories, Tool-Calls, Task Completion

You've built the agent: a model in a loop, with tools, that remembers. Now the question that decides whether it ever ships — is it any good, and how would you know? The instinct is to check the final answer. For an agent, that instinct is a trap. The answer can be right while everything that produced it was wrong, and the only way to catch that is to grade the path, not just the destination.

1Why a right answer can still be a failure

The first three articles built the agent. This one asks whether it works — and the trap is that "works" feels obvious until you try to measure it. For a plain LLM app, you mostly can trust the answer: one input, one output, check it against a reference. For an agent, the answer is the last step of a journey, and the journey is where everything goes wrong. So the objective of this section is to break the instinct you brought from LLM eval before it costs you a production incident.

Picture a support agent asked to refund an order. It calls lookup_order, then issue_refund, then replies "Done — your refund is processing." Final answer: correct. Ship it? Now look at the path: it issued the refund before checking the return policy, refunded the wrong amount, and only landed on the right total because this particular order happened to be a flat-rate item. The answer is right by luck. Run it on the next order and it refunds $400 against a $40 return.

A plain LLM gives you an answer to grade. An agent gives you a story, and the answer is only its last sentence.

This is the whole reason agent eval is its own discipline. The thing you're grading isn't a string — it's a trajectory: the ordered sequence of the model's reasoning, the tools it chose, the arguments it passed, the observations it got back, and the final response. A grader that reads only the last line is blind to wrong-order tool calls, unnecessary steps that triple the cost, a tool called with a hallucinated argument that happened not to matter this time, and the silent near-miss that becomes next week's outage.

If you come from software testing, there's a familiar analogy that makes the shift click. A unit test checks one function in isolation: format_currency(40) returns "$40.00", deterministically, every time. That's the LLM-app instinct — pin one input to one output. Trajectory eval is the other kind of test entirely: it checks whether the agent, handed a goal it has never seen, strings the right operations together in the right order — and recovers when a tool throws an error mid-run and it has to fall back to another. You're not asserting a return value; you're grading emergent behavior over a path. Bring only unit-test instincts to an agent and you'll verify the pieces while staying blind to how they're assembled.

2The three layers of agent eval

If §1 convinced you the answer alone isn't enough, the natural next question is: then what do you measure? The field has converged on three layers, and the goal of this section is to give you all three as a single mental checklist — because the most common failure in practice is measuring only the first and calling it evaluation.

The layers aren't ranked — they catch different failures, and an agent has to pass all three to be production-worthy. An agent that completes the task but takes a wild path passes layer 1 and fails layer 2: it works today and breaks the moment the inputs shift, exactly the refund agent from §1. An agent that takes a clean path but costs $50 a query passes layers 1 and 2 and fails layer 3 — correct, elegant, and unshippable. Only watching one layer means flying blind on the other two.

3Choosing a scorer: deterministic vs. LLM-as-judge

Knowing the three layers tells you what to measure; it doesn't tell you how to put a number on something as fuzzy as "did it take a sane path." That's the job of a scorer, and almost every scorer is one of two kinds. Picking the wrong one is how eval suites end up either brittle or meaningless, so this section is about matching the scorer to the shape of the question.

Deterministic checks, for things with one right answer

When a question has a single verifiable answer, you don't need a model to grade it — you need code. "How many customers are from Canada?" has exactly one correct response (eight), and a string or numeric match settles it: fast, free, perfectly repeatable, and immune to a judge's mood. The same applies on the trajectory layer when you know the path: assert that issue_refund was called after check_policy, with the right arguments, and you've caught the §1 bug deterministically. Reach for this whenever the correct behavior is knowable in advance — it should be your default, because it's cheap and it never lies.

LLM-as-judge, for things only a human could grade

But many agent outputs have no single right form. "Which employee generated the most revenue, and from which countries?" has a correct core, yet a dozen equally valid phrasings — and qualities like completeness, clarity, and tone that a string match can't see. Here you hand the agent's output (and optionally a reference) to a second model acting as judge, with a rubric: score correctness, completeness, and clarity from 1–5. It's flexible enough to grade open-ended work and nuanced trajectory questions like "was this the efficient path?" — at the cost of an extra LLM call and some non-determinism. Use it where deterministic checks can't reach, not as a lazy default.

4Scoring the trajectory in practice

Sections 2 and 3 were the concepts: three layers, two scorer types. Now we make the hardest layer — the trajectory — concrete, because "grade the path" stays abstract until you see the two ways it's actually done. Both come from the open-source agentevals package, and the choice between them is the §3 distinction applied to paths: do you know the right path in advance, or not?

Trajectory match, when you know the right path

For a well-defined workflow, you hard-code the reference trajectory — the exact sequence of tool calls you expect — and compare the agent's run against it step by step. No LLM needed: it's deterministic, fast, and cheap. This is the right tool for testing that a known workflow stays on its rails, and for catching the wrong-order bug from §1 as a hard failure.

LLM-as-judge on the trajectory, when the path is open

When there's no single correct path — the agent could reasonably solve the task several ways — you drop the reference and let a judge assess the trajectory against a rubric: was the tool selection appropriate, the order sensible, the work free of pointless detours? More flexible, costs a call, and doesn't need you to enumerate every valid route. Here are both, from agentevals:

# Hard-code the expected path; compare the run step by step.
# Deterministic — no LLM call, no flakiness.
from agentevals.trajectory.match import create_trajectory_match_evaluator

evaluator = create_trajectory_match_evaluator(
    trajectory_match_mode="superset",   # strict | unordered | superset | subset
)

result = evaluator(
    outputs=agent_messages,            # what the agent actually did
    reference_outputs=expected_messages,  # the path you expect
)
# -> {"key": "trajectory_match", "score": True/False, ...}
# mode picks how strict: exact order, any order, allow extra steps, etc.

# No reference path. A judge scores the trajectory against a rubric.
# Use when several routes are legitimately correct.
from agentevals.trajectory.llm import create_trajectory_llm_as_judge

judge = create_trajectory_llm_as_judge(
    model="anthropic:claude-sonnet-4-5",   # the grader; pin temperature low
)

result = judge(
    outputs=agent_messages,            # reference_outputs optional here
)
# -> the judge reviews tool choices, order, and efficiency,
#    then returns a score + written justification you can audit.

# Wire either evaluator into pytest so eval runs on every PR.
# A dropped score fails the build — eval becomes a gate, not a vibe.
import pytest
from langsmith import testing as t

@pytest.mark.langsmith
def test_refund_trajectory():
    agent_messages = run_agent("Refund order 9847")
    result = evaluator(
        outputs=agent_messages,
        reference_outputs=expected_messages,
    )
    assert result["score"] is True   # policy checked BEFORE refund issued
# Runs in CI alongside unit tests; set a threshold, fail below it.

5The reliability trap: pass@k vs pass^k

Every scorer so far produces a pass or fail on one run. But agents are non-deterministic — the same input can take a different path each time — so a single run tells you what happened once, not what to expect. This section is about the metric that exposes that gap, because it's the one most teams get wrong, and getting it wrong means shipping an agent you believe is reliable when it isn't.

Two metrics, opposite behavior. pass@k asks: across k attempts, did the agent succeed at least once? It measures the capability ceiling — given enough tries, can it ever get this right? pass^k (pronounced "pass-hat-k") asks the stricter question: did it succeed on all k attempts? That measures reliability — can you count on it every single time? The two scale in opposite directions: as k grows, pass@k rises toward 100% while pass^k falls.

The gap is not a rounding error. Take an agent that succeeds 70% of the time on a single run:

The same agent is "97% good" and "34% reliable." Which number you report decides whether you ship a disaster.

For an airline rebooking thousands of flights a day, the 34% is the truth that matters: handle three consecutive requests and there's only a one-in-three chance all three go right. pass@k is the honest metric when you can verify the output before using it and just need one working result — code generation against unit tests is the classic case. pass^k is the honest metric when a single failure is costly and the agent can't self-check — customer-facing work, anything in a compliance path. Report pass@k for a customer support agent and you are measuring the wrong thing on purpose.

6Offline gates vs. online monitoring

Sections 3–5 gave you scorers and a reliability metric. The last structural question is when they run — and there are two answers, not one, because the eval that protects you before a release is different from the one that protects you after. This section places both on the agent's lifecycle so neither gap goes unwatched.

Case A: offline evaluation — the pre-merge gate

Offline eval runs a fixed, versioned dataset of test cases against the agent before you ship. Each case has known success criteria; the scorers from §3–§4 grade the run; you set a threshold and fail the build below it. Wired into pytest and CI (the §4 gate), this brings unit-test rigor to a non-deterministic system: every PR is checked against the same cases, regressions are caught before merge, and "did this change make the agent worse?" gets a number instead of a hunch.

Mock the tools, or the gate isn't reproducible

There's a catch that quietly breaks offline eval, and it follows straight from pass^k's "deterministic checks only" rule in §5. If your test run lets the agent call its real tools, two bad things happen: the run fires real side effects — it actually issues the refund, sends the email, mutates the row — and it becomes non-reproducible, because a live API returns different data each run and your trajectory match flaps. A flaky harness fails honest agents, and a destructive harness can't run in CI at all.

The fix is to swap the tool layer for a deterministic mock during eval: same tool name and schema the agent expects, but a fixed canned response instead of a live call. The agent's reasoning and tool selection are still exercised for real — you're testing the agent — while the environment underneath is frozen, so the same input produces the same trajectory every time and nothing real gets touched. Mock the tool, not the agent:

# The real tool hits a live API — non-deterministic, and it really
# issues the refund. For eval, swap in a mock with the SAME signature.
from unittest.mock import patch

def fake_lookup_order(order_id: str) -> dict:
    # fixed fixture — identical every run, no network, no side effects
    return {"id": order_id, "total": 40.00, "returnable": True}

def fake_issue_refund(order_id: str, amount: float) -> dict:
    # records the call so we can assert on it — but refunds NOTHING
    return {"status": "ok", "refunded": amount}

with patch("app.tools.lookup_order", fake_lookup_order), \
     patch("app.tools.issue_refund", fake_issue_refund):
    agent_messages = run_agent("Refund order 9847")
# Same fixtures every run -> identical trajectory -> pass^k is meaningful.

# The mock makes the §4 trajectory gate reproducible AND safe to run in CI.
import pytest
from langsmith import testing as t

@pytest.mark.langsmith
def test_refund_path_is_safe():
    with patch("app.tools.lookup_order", fake_lookup_order), \
         patch("app.tools.issue_refund", fake_issue_refund):
        agent_messages = run_agent("Refund order 9847")

    result = evaluator(                       # the §4 trajectory-match evaluator
        outputs=agent_messages,
        reference_outputs=expected_messages,
    )
    assert result["score"] is True   # policy checked BEFORE refund — every run, no live calls
# Deterministic environment + deterministic check = a gate you can trust.

Case B: online evaluation — the production watch

But your dataset only contains failures you already knew to write down. Production sends inputs you never imagined, and the world drifts — a tool's API changes, user phrasing shifts, a model update lands. Online eval runs judges on sampled live traffic, scoring real trajectories as they happen and surfacing the failures the dataset never anticipated. The highest-value move closes the loop: every production failure you catch becomes a new case in the offline dataset, so the same bug can never ship twice.

7The evaluation checklist

The diagram-level picture is complete; this section turns it into something you can run an agent against before shipping. Each row distills one decision from the sections above into a single rule.

Conclusion: you can't ship what you can't measure

The first three articles built an agent — a model in a loop, with tools, that remembers. This one answered the question that gates all of it: how do you know it's any good? The answer is that you stop grading agents the way you graded LLM apps. A plain model hands you an output to check. An agent hands you a trajectory, and the output is only its last step — so you grade the path: the tools it chose, the order it chose them in, the arguments it passed, and the cost of getting there.

That splits into a discipline. Three layers, because outcome alone hides the broken paths and the runaway costs. Two scorers, because some answers are knowable and some only a judge can grade — and the judge is software you test too. And one metric most teams get wrong: pass^k, not pass@k, whenever a single failure is costly, because capability and reliability are different numbers and the gap between them is where production disasters live. Gate with an offline dataset, watch with online judges, and feed every failure back so the same bug can't ship twice.

This is the discipline that turns "the demo worked" into "the agent is reliable enough to deploy." The model brings the reasoning; the loop, the tools, and the memory make it act and persist. Evaluation is what earns it the right to run unattended.