Fetching from the wire…
01
02
03
04
05
06
07
08
Skills2026-03-29 · source-backed
Measure four dimensions: consistency (same task, same result), robustness (degraded conditions), predictability (calibrated uncertainty), and safety (bounded error severity). An agent at 90% success with unpredictable 10% failure is an assistant, not an autonomous system. Know the difference before your users discover it.
Each link below shares sources, entities, or timing with this story.
A single passing eval hides nondeterminism. Measure all-runs consistency with pass^k (the probability all k runs pass) so flaky agents get caught before release. Pair it with a judge calibrated against a human gold set, and grow that golden set from real production traces so t...
Kuber Mehta's argument is that brevity and ADHD-friendly instructions compress and hide information as agents pass state to each other. An intermediate step should emit 5/6 PASS, FAIL: test_cache_invalidation, not a friendly summary that masks the failure and invites hallucina...
Data that contradicts the vibe. That's rare enough to lead with. Dipongkor, Baral, Lam and Moran analyzed 4,882 pull requests from five coding agents in the AIDev dataset (532 Java, 4,350 Python), accepted to ICSME 2026. The findings, in order of how much they should change yo...
Somebody finally measured the thing everyone complains about, and the numbers are worse than the vibes. A Level1Techs writeup that hit 384 points and 144 comments on Hacker News captured full-vocabulary logits and computed KL divergence in FP64 to trace exactly where local inf...
40+ experts and 31,000+ human hours built 60 project-level research tasks across 11 domains, designed so guidance is progressively withdrawn *within the same project*. Across 18 agent-model configurations: 50.91 with full methodological guidance, 29.10 when only the method is...
AAS Core hit v15.6.0 with 1,993 skills and 44,119 stars, and it explicitly does not rank or recommend. The agent inspects the project, searches the complete local catalog over a read-only stdio MCP server, picks exact skill IDs, and a read-only compose_stack tool validates tha...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.