Fetching from the wire…
01
02
03
04
05
06
07
08
Skills2026-08-07 · source-backed
CodeGrep found BM25 at 0.375 precision actively degrades agent performance, Jina at 0.445 is neutral, and only 0.677 helps. Below roughly 0.45 precision you're paying tokens to make the agent worse. Test on your own repo before you assume retrieval is free upside.
Each link below shares sources, entities, or timing with this story.
CodeGrep measures a 30B OpenHands agent averaging 23 rounds and 631K tokens per resolved SWE-Bench Verified issue, much of it grep, glob and view_file. A 14B retrieval agent trained end-to-end with GRPO raises resolve rate to 27.0% from 25.8% while cutting 15% of rounds and 19...
SpecPath found 35 of 100 passing implementations broke when only the revision path changed, with aggregate accuracy looking identical across paths. Build your eval set from real multi-turn clarification threads with amendments and reversals. Path sensitivity is invisible to st...
ExecRetrieval builds a search pool containing the counterfactuals: 939 Python tasks, each with one execution-verified canonical implementation and up to four execution-verified buggy variants from a single targeted mutation. Across 23 dense embedding configurations plus BM25,...
SpecFirst splits the loop in two: a spec agent probes the binary and fuses observations with documentation into a structured specification, then a separate synthesis agent codes against that fixed reference. Test pass rates rose 6.9-21.3% and binary exploration coverage 9.4-18...
RAGAS-style evaluation checks correctness against a frozen snapshot, which means routine document updates and corrections can silently break production without moving a dashboard. This ASE 2026 paper defines 11 mutation operators perturbing at both the pre-chunk index level an...
It separates three claims usually bundled together, that an expert file supplies hard-to-find knowledge, produces a recognizable persona, and changes what the agent decides, and tests each (arXiv 2609.00453). Eight logged builds averaged 38 model calls, and the quotation check...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.