Fetching from the wire…
Public story · 2026-07-30 · high
It logs each failure's trigger and fix, then rewrites only that record, leaving the agent's tools and context untouched.
Why now: This lands in the July 30 coverage simply because that's when the preprint surfaced, not because the failure mode itself is new.
Living-Harness rewrites an AI agent's setup after each finished task so a fixed bug doesn't return, per a paper posted to arXiv.
The paper targets one failure: an agent fixes an error mid-task, then repeats it on a later task because nothing carried the fix forward. For teams running agents across repeated tasks, that gap is the cost, not the model's raw capability.
The fix works through what the paper calls an Evolution-SOP, a domain-level process for updating two things after each run. One is episodic memory: what triggered the failure, what the failure pattern looked like, and which recovery action worked. The other is a state graph, built from nodes, repair edges, and transition rules, that routes the agent around the same trap next time.
Tools and the agent's base context stay fixed throughout. Only the procedural layer changes: no bigger model, no new toolset, just accumulated repairs. The authors tested this on eight environments from τ²-Bench and MultiWOZ-2.4, established agent and dialogue benchmarks, rather than building a new one for the paper.
Each link below shares sources, entities, or timing with this story.
HarnessOpt-Bench (arXiv 2608.06301) has a frontier LLM act as an optimizer receiving a target agent's seed harness (prompts, tools, control flow, memory, orchestration code) plus graded eval feedback and a fixed evaluation budget, then edits and nominates a candidate scored on...
Deng et al. built 120 real-case-grounded tasks across 20 business scenes in six financial domains, running four self-evolving scaffolds on a shared Qwen3.7-Max backbone against paired non-evolving controls. Letta posted the highest evolved score (91.65) and fewest compliance i...
It treats the executable runtime, context construction, tool mediation, action validation, execution recovery, as the thing to learn. A separate harness engineer converts batches of target-agent failures into validated executable patches, with same-batch reruns of the frozen t...
Here's the number that should sit in every "agents will replace engineers" thread: 15.2%. That's the best model. The mean across 15 frontier models is 4.3%. Tencent Hunyuan's Long-Horizon-Terminal-Bench put 15 frontier models against 46 long-horizon terminal tasks across nine...
"Remember When It Matters" (arXiv:2607.08716) attacks behavioral state decay, where task-critical instructions get buried or evicted on long runs, using a second memory agent that proactively injects reminders into the action agent's context. It gained +8.3pp pass@1 on Termina...
ByteDance Seed's GST-Bench covers 6,790 minutes of synthetic video with human-verified questions, isolating a specific failure: models handle local spatial relations competently but can't consolidate long-horizon observations into a globally consistent scene. The ~36-point gap...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.