Outcome-First Agent Grading: Grade Final Environment State, Not Tool-Call Sequence

Anthropic Engineering·high signal

Anthropic's eval framework establishes that grading agent transcripts (which tools were called, in what order) creates false failures when agents find valid alternate paths. The correct primitive is outcome grading — verify the final state in the environment (does the database record exist?) independent of how the agent got there. A flight-booking agent can verbally confirm success while failing the booking; only environment-state graders catch this, and building separate transcript vs. outcome eval layers exposes previously invisible failure modes.