Evaluating Trajectories

One Desk trial finishes with the correct refund in the payment system. One metric scores that trial 0, a second scores it 1, and a third scores it 0.83. All three calculations are correct, and all were applied to the same six tool calls.

The metrics disagree because each asks a different question. A path score reports whether the agent matched a path that somebody wrote down, which is separate from whether the agent did well.

What Trajectory Evaluation Measures

**Trajectory evaluation scores the sequence of tool calls in a trial, along with the arguments and the use of each result