Evolution Strategies beat GRPO on Pass@K without entropy collapse.

ES achieves broader reasoning coverage, with verifier-projected Jensen-Shannon diversity across the ES population theoretically tied to higher Pass@K, and empirically improves Pass@1 while reaching higher Pass@K where GRPO collapses entropy. The proposed sequential GRPO-then-ES schedule keeps GRPO's Pass@1 and adds ES's Pass@K. Two side findings: despite large whole-model parameter drift, gains come from a sparse subset of larger-magnitude updates, and larger LLMs need a smaller ES population. (arXiv)