One in Seven Python Samples That Pass Bandit and Semgrep Still Carries a Runtime-Confirmed Exploit
arXiv·medium signal
arXiv 2609.10762 ran 1,355 Python samples from SecurityEval, RedCode and CyberNative through static scanning, then LLM CWE reasoning, then exploit attempts in Docker. Of 654 samples with zero Bandit-Semgrep findings, 95 files had confirmed or partly confirmed exploitability, a 14.53% static-pass dynamic-fail rate. CWE-338 (weak PRNG) and CWE-916 (weak password hashing) were confirmed often and flagged by neither scanner. A clean SAST run on AI-generated code is not a security sign-off.