Judge long-form agent outputs by decomposing into atomic claim checks — holistic single scores are unreliable at length

arXiv 2606.01629·medium signal

A June 2026 arXiv benchmark on judging long-form output finds that holistic single-score LLM judging becomes unreliable as length grows. The fix is to decompose evaluation into atomic claim or criterion checks and aggregate, rather than asking the judge for one overall score on a long document. If you evaluate long agent reports or generations, switch to checklist-style per-claim rubric scoring to recover judge reliability and surface exactly which claims fail.