Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-02 · source-backed
IFHierBench targets one call producing a layered artifact where the whole output, each section, and nested fields all carry constraints: something flat benchmarks can't score. 600 prompts across four constraint-tree depths and 35 constraints, each with a deterministic checker verifying every scope. Accuracy degrades sharply as depth increases. Flatten your constraint tree or split deep requirements across calls.
Each link below shares sources, entities, or timing with this story.
VAKRA (arXiv 2608.12282) benchmarks agents against 8,000+ executable APIs across 62 domains, verifying by re-executing predicted calls against live endpoints. Accuracy falls to 50-51% on compositional APIs and degrades over 50% as depth grows. Failures concentrate in entity di...
A 1.5B distilled model trained with GRPO chooses NoThink, Short, or Long at response start, using a shaped reward that makes each mode pay off at a different length plus hard per-mode token caps. Accuracy held at 0.782 against 0.796 baseline while mean length fell from 4,796 t...
arXiv 2607.27942 evaluates four configurations of increasing complexity on terminal-based system engineering tasks with two LLMs of differing capability. Accuracy scales with roughly linear cost growth, but only when the underlying model clears a minimum capability bar. Past i...
arXiv 2607.06283 attacks the problem that growing skill libraries make selection harder. It decomposes on both the task and skill side, builds a DAG with intermediate task states as nodes and candidate skills as edges, then cross-encodes over candidates per task interval. On A...
ArXiv paper 2603.17683 benchmarks a deceptively simple idea: instead of stuffing facts into your agent's context window and hoping the model remembers them, treat each fact as a discrete, hash-addressed tuple stored externally and retrieved on demand. The results are stark: 10...
Treat each fact as a content-hashed tuple with typed metadata and a retrieval interface. Retrieve relevant subsets on demand instead of stuffing everything into context. 100% accuracy at 7,000+ facts where in-context approaches lose 60%. The database approach to LLM memory. arXiv
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.