AI Planning Framework for LLM Web Agents: Five Trajectory Quality Metrics Beyond Binary Success Rate

arXiv 2603.12710·medium signal

arXiv 2603.12710 introduces five evaluation metrics for assessing intermediate trajectory quality in LLM web agents, accompanied by a new dataset of 794 human-labeled WebArena trajectories. Existing benchmarks only capture task completion, missing reasoning quality at each step — this framework enables finer-grained diagnosis of where agents fail during multi-step web navigation. Directly applicable to teams building or evaluating autonomous browser/web agents.