Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
An 8,135-trial study finds skill files mostly lock in a procedure, and a 100-item skill pool nearly kills retrieval accuracy.
Why now: The paper posted to arXiv on August 14, 2026.
Agent skills work mainly by locking in a stable execution path, not by teaching new facts, per an arXiv study of 8,135 trials posted August 14, 2026. Procedural anchoring accounts for 65.7% of the measured benefit, versus 4.5% for explicit knowledge injection. That's the opposite of what most SKILL.md files are written to do.
The paper is titled "Demystifying Agent Skills: Why They Work, Until They Don't," by Zhiyuan Jiang and coauthors. It normalized 8,135 trial records into 238 labels across 12 skill-use modes.
In matched comparisons, per the paper, skills beat Workflow Memory by 6.06 points. That backs the anchoring finding: a fixed procedure beats a stored memory of past runs.
The failure mode sits in retrieval, not content. Growing the skill pool from 5 to 100 items drops actual-use precision from 29.6% to 3.3%, per the paper's trial data. A near-collapse, not a gradual decline, and it hits anyone stockpiling skills expecting the agent to find the right one.
I've been building with Claude Code in my personal projects long enough to collect a pile of half-used skill files myself. This gives me a number to point at instead of a hunch. If your agent has dozens of skills sitting around and results have gotten worse, check retrieval first, not the content.
Skill libraries that grow past a few dozen items will fail on retrieval before they fail on content. Most teams are optimizing the wrong 4.5% of the benefit. Watch whether builders respond by capping skill pools or by building retrieval that works at scale. The paper's own numbers argue for the cap: a small, curated skill set beats a sprawling library an agent can't find.
Each link below shares sources, entities, or timing with this story.
The attack hides malicious intent across separate skills that only turn dangerous when they pass work to each other, and a fix cuts success to 22.5%.
Attackers who know only a target's role profile can chain marketplace skills into working attacks; success drops off after three hops.
Four defenses were tested, one per distillation method, and every one failed when the method changed.
Adding a third label instead of forcing human-or-bot gives every AI agent a perfect detection score, because Playwright never generates real pointer telemetry.
Testing eight models across 192,000 evaluations, researchers found chain-of-thought and direct instructions to ignore the score didn't remove the bias.
A 4,181-problem study found confidence-based escalation beats a frozen router by 4.2 points while using 37% fewer tokens.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.