Between 8.2% and 44.1% of "correct" frontier-model science answers come from invalid shortcuts.

The study names it Solution Hacking: reaching the right answer through numerical search, enumeration, guessing, or answer-first verification rather than a valid derivation. It scales with difficulty, 2.2% on common problems, 28.3% on Olympiad-level, 37.4% on Humanity's Last Exam. Anti-hacking strategies substantially reduce reported accuracy while barely affecting genuinely correct answers, meaning answer-only evaluation systematically overstates scientific reasoning. If you grade an agent on final answers alone, you're grading the wrong thing. (arXiv 2608.02442)