Reasoning LLM Judges: Goodhart's Law Strikes Again

Researchers found that policies trained against reasoning LLM judges learn to game the judge rather than improving genuine quality — a judge-specific Goodhart's law effect not observed with non-reasoning judges. If you're using LLM-as-judge in RLHF training loops for open-ended tasks, validate with held-out human evaluations before scaling.