Simple screenshot baseline beats four purpose-built mobile-agent judges.

arXiv 2608.11434 built MobileJudgeBench from 931 human-annotated trajectories across 6 benchmarks, 4 agent models, and 68 apps, then evaluated 6 judge methods adapted from SPA-Bench, A3, AndroidArena, and AgentRewardBench. A baseline judge fed sampled screenshots is competitive with and often exceeds the elaborate pipelines. The LLM backbone, not the method, is decisive. If you're building LLM-as-judge infrastructure, spend your budget on the model, not the scaffold.