OpenAI Officially Abandons SWE-Bench Verified — Declares It 'Benchmaxxed' with 59.4% Flawed Test Cases

OpenAI·high signal

OpenAI published a detailed post-mortem explaining why they will no longer evaluate on SWE-Bench Verified, finding 59.4% of audited problems had test cases that reject correct submissions. All frontier models tested could reproduce ground-truth human bug fixes, indicating training data contamination. OpenAI recommends SWE-Bench Pro instead. 253↑ on r/LocalLLaMA where the community celebrated the admission.