Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-06 · source-backed
arXiv 2608.04682 removes the assumption that every SWE benchmark makes, that a high-quality issue report exists. Six bug categories, eight languages, multi-bug fixing and potential-bug discovery under dual-track evaluation. Most state-of-the-art coding agents perform poorly at locating recorded bugs without report guidance, handling multi-bug scenarios, and surfacing valid potential bugs. That's the gap between SWE-bench-style scores and what happens when you point an agent at a repo with no ticket, which is most of real work.
Each link below shares sources, entities, or timing with this story.
The first benchmark built on continuous integration loops evaluates agents on long-horizon codebase maintenance (average 233 days, 71 consecutive commits per task). Most models achieve a zero-regression rate below 0.25 — only Claude Opus exceeds 0.5. Even frontier agents that...
A team at UC Berkeley RDI built an automated scanning agent that achieved near-perfect scores on eight major AI agent benchmarks. SWE-bench Verified: 100%. Terminal-Bench: 100%. WebArena: approximately 100%. FieldWorkArena: 100%. GAIA: roughly 98%. OSWorld: 73%. The agent didn...
The FSE '26 paper argues SWE-bench, SWT-bench, and AgentBench capture narrow synthetic slices, and proposes contamination-aware, trajectory-aware, in-the-wild evaluation using agents' commit signatures to study real vs human contributions over time. (arXiv) Pair this with the...
While the capability stories pile up, here's the counterweight. As SWE-bench Verified scores cluster near saturation on July leaderboards, an enhanced analysis (SWE-Bench+, on the AIware 2026 benchmark track) found 60.83% of commonly resolved issues contain solution leakage ri...
Traced across 557 SWE-chat sessions (94,813 events) and 33,097 agentic pull requests from AIDev. Agent-facing artifacts account for 60.5% of documentation interactions versus 10.6% for classical technical docs and 1.3% for API references. Consultation is self-initiated 70.2% o...
Agent Lightning v1.0 (arXiv 2608.17528) inverts the standard agentic RL architecture, and the inversion is the whole point. Normally the training engine owns the environment loop. It drives the agent, collects trajectories, computes rewards. Which means your training setup and...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.