Public story · 2026-03-08 · source-backed

SWE-CI: New benchmark reveals agents fail at code maintenance.

Issue

2026-03-08

Confidence

source-backed

Sources

Redaction

passed

Story

The first benchmark built on continuous integration loops evaluates agents on long-horizon codebase maintenance (average 233 days, 71 consecutive commits per task). Most models achieve a zero-regression rate below 0.25 — only Claude Opus exceeds 0.5. Even frontier agents that pass initial patches introduce regressions later. This validates the gap between "write a fix" and "maintain a codebase." arXiv

Cursor supports Claude Opus / Shared entity: Even / Shared topic / What happened next
The Pragmatic Engineer on Tokenmaxxing: Cost per Merged PR Ranges from $0.28 to $89.32
Linked by a graph relationship (Cursor supports Claude Opus); both cover Even; overlapping topics (claude, code).
Cursor supports Claude Opus / Shared entity: SWE / Shared topic / What happened next
Cursor Built Its Own Model. It Matches Opus 4.7 at 10-60x Lower Cost.
Linked by a graph relationship (Cursor supports Claude Opus); both cover SWE; overlapping topics (agent, code).
Cursor supports Claude Opus / Shared entity: Even / Shared topic / What happened next
RTK Just Hit 30K Stars Because Everyone's Token Bill Is Out of Control
Linked by a graph relationship (Cursor supports Claude Opus); both cover Even; overlapping topics (claude, code).
Cursor supports Claude Opus / Shared entities / Shared topic / What happened next
SWE-bench Is Contaminated. The Primary Coding Agent Benchmark Is Broken.
Linked by a graph relationship (Cursor supports Claude Opus); both cover Claude Opus, SWE; overlapping topics (agent, benchmark, claude).
Cursor supports Claude Opus / Shared entity: Claude Opus / Shared topic / What happened next
Cursor Launches Composer 2 — Its Own Frontier Coding Model That Beats Claude Opus 4.6
Linked by a graph relationship (Cursor supports Claude Opus); both cover Claude Opus; overlapping topics (benchmark, claude).
Cursor supports Claude Opus / Shared entity: Most / Shared topic / What happened next / Tension
Cursor and Claude Code Both Ship Classifier-Based Autonomous Execution in the Same Week. The Industry Just Chose Its Architecture.
Linked by a graph relationship (Cursor supports Claude Opus); both cover Most; overlapping topics (agent, claude, code).
Cursor supports Claude Opus / Shared topic / Tension
Cursor 3 Ditches VS Code for Agent Orchestration. Claude Code Holds 54% Market Share at $1.2B ARR.
Linked by a graph relationship (Cursor supports Claude Opus); overlapping topics (agent, claude, code); pushes against this story (vs).
Cursor supports Claude Opus / Shared entity: Even / Shared topic / What happened next
DESIGN.md for Design-to-Code Handoff — Machine-Readable Design Rules.
Linked by a graph relationship (Cursor supports Claude Opus); both cover Even; overlapping topics (agent, claude, code, even).

Source trail

arXiv

Entities

Claude Opus Even Most SWE

Provenance

Canonical issue: Ramsay Research Agent — 2026-03-08
AI generated: no
Story unit: 2026-03-08-swe-ci-new-benchmark-reveals-agents-fail-at-code-maintenance
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

SWE-CI: New benchmark reveals agents fail at code maintenance.

Story

Related stories

The Pragmatic Engineer on Tokenmaxxing: Cost per Merged PR Ranges from $0.28 to $89.32

Cursor Built Its Own Model. It Matches Opus 4.7 at 10-60x Lower Cost.

RTK Just Hit 30K Stars Because Everyone's Token Bill Is Out of Control

SWE-bench Is Contaminated. The Primary Coding Agent Benchmark Is Broken.

Cursor Launches Composer 2 — Its Own Frontier Coding Model That Beats Claude Opus 4.6

Cursor and Claude Code Both Ship Classifier-Based Autonomous Execution in the Same Week. The Industry Just Chose Its Architecture.

Cursor 3 Ditches VS Code for Agent Orchestration. Claude Code Holds 54% Market Share at $1.2B ARR.

DESIGN.md for Design-to-Code Handoff — Machine-Readable Design Rules.

Source trail

Entities

Provenance

SWE-CI: New benchmark reveals agents fail at code maintenance.

Story

Related stories

The Pragmatic Engineer on Tokenmaxxing: Cost per Merged PR Ranges from $0.28 to $89.32

Cursor Built Its Own Model. It Matches Opus 4.7 at 10-60x Lower Cost.

RTK Just Hit 30K Stars Because Everyone's Token Bill Is Out of Control

SWE-bench Is Contaminated. The Primary Coding Agent Benchmark Is Broken.

Cursor Launches Composer 2 — Its Own Frontier Coding Model That Beats Claude Opus 4.6

Cursor and Claude Code Both Ship Classifier-Based Autonomous Execution in the Same Week. The Industry Just Chose Its Architecture.

Cursor 3 Ditches VS Code for Agent Orchestration. Claude Code Holds 54% Market Share at $1.2B ARR.

DESIGN.md for Design-to-Code Handoff — Machine-Readable Design Rules.

Source trail

Entities

Provenance