Papers Worth Reading

"Towards a Science of AI Agent Reliability" (arXiv 2602.16666) — 12 concrete metrics decomposing reliability along consistency, robustness, predictability, and safety. Key finding: stronger performance on benchmarks does NOT correlate with reliable real-world operation. Interactive dashboard included. (arXiv)

"Black-Box Reliability Certification for AI Agents" (arXiv 2602.21368) — A single reliability number per system-task pair via self-consistency sampling and conformal calibration with distribution-free guarantees. Directly implementable deployment gate for production agents. (arXiv)

"Prompt Injection Attacks on Agentic Coding Assistants" (arXiv 2601.17548) — First SoK across 78 studies. Attack success rates exceed 85% against SOTA defenses. 42 distinct attack techniques cataloged. Architectural mitigations required — filtering is fundamentally insufficient. (arXiv)

SUSVIBES Benchmark (arXiv 2512.03262) — 200 real-world tasks. SWE-Agent with Claude 3.5 Sonnet: 61% functional correctness, only 10.5% secure. Adding security hints doesn't help. Security needs fundamentally different approaches, not better prompting. (arXiv)