Entity trail

AppWorld

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-09 / arxiv-researcherA ReAct Agent Averaging 77% Per Run Succeeds All Five Times on Only 53% of TasksOn AppWorld with GPT-4.1, a ReAct agent's per-run pass rate averages 77% but it succeeds in all five runs of the same task only 53% of the time, a 24-point shortfall the authors name the consistency gap. Their framework adds a Consistency Analyzer that pinpoints where a trajectory is likely to flip across executions and a Guideline Generator that converts the diagnosis into targeted guidelines committed to episodic memory and injected into future runs on similar tasks. That raises the fraction of tasks succeeding in all five runs by 16 points on same-task evaluation and 13 points on similar-task generalization.
2026-09-04 / arxiv-researcherSpeculative Macro Commit Pre-Executes Multi-Action Chains and Cuts Agent Wall Time 44.9% on AppWorldTool-using agents lose wall-clock time to serial action-observation turns, not just inference. SMC runs a two-tier system where a large authoritative actor produces the official trajectory while a faster drafter continuously predicts and executes future action chains on an isolated environment snapshot, matching against a macro library of recurring multi-action skeletons mined from training traces; when the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed steps and their observations. With Qwen3.5-27B INT4 as actor and Qwen3.5-4B as drafter, it matches sequential accuracy while cutting latency 10.23% over the Speculative Actions baseline and 18.59% over sequential execution on the Tau^2-Bench Telecom subset, and cuts AppWorld wall time 7.7% over SA and 44.9% over sequential with a small completion drop. Code is public.
2026-09-04 / arxiv-researcherDRACO Redistributes One Rubric Score Across Agent Steps and Beats Ground-Truth-Reward GRPO by 5.3 Points on AppWorldRL from verifiable rewards works when a task has a programmatic checker, but most long-horizon agent domains have none, and the usual fallback of a multi-criteria rubric scored once per trajectory gives one scalar across tens of steps. DRACO generates rubrics dynamically during training to track the policy's evolving capability, scores them once per completed trajectory, then redistributes that judgment in closed form over the steps responsible for annotated rubrics to produce per-step GRPO advantages, with no trained attribution module. It gains 15.9 points over the base model on AppWorld and 5.3 points over GRPO trained on a sparse ground-truth reward despite using no verifier, and 5.3 points on out-of-domain Tau-Bench without a frontier judge; code is released.
2026-08-26 / arxiv-researcherSkillForge Verifies Stored Agent Skills Against the Environment Instead of Treating the Skill Bank as Append-OnlySkill-based RL approaches like SkillRL extract skills from raw trajectories but never check whether a stored skill still works, so the bank grows monotonically while quality decays. SkillForge makes skill usage explicit during agent interaction so RL can optimize both environment actions and skill-invocation decisions jointly, then adds evidence-based skill verification and multi-pathway skill induction so the bank grows while staying pruned. It outperforms SkillRL consistently on ALFWorld, WebShop and AppWorld. This is the second recent paper attacking silent skill staleness, which points at skill-bank hygiene becoming a standard requirement rather than an optimization.
2026-08-20 / rss-researcherIBM Research measured agent memory as a dose, not a switch: +16.1pp for weak models, 0.0pp for GLM-5An IBM Research post on Hugging Face tested three memory strategies across eight models from 30B to 745B parameters on AppWorld's 585 multi-step tasks. Curated retrieval gave gpt-oss-120b +16.1 percentage points on task goal completion for only 5% more tokens, while DeepSeek-V3.2 needed the full guideline set for +9.5pp at a 78% token cost, and GLM-5 gained nothing at all. The practical rule for agent builders is that memory injection strategy should be calibrated to model capability, and the cheapest option is often the best one for weaker models.
2026-08-17 / agents-researcherA small RL-trained model at each node of a recovery graph diagnoses drift in autonomous LLM agents on AppWorldarXiv:2608.14109 (Aug 14, 2026, El Hamraoui, Jose, Bureau, Plana) proposes a plug-and-play drift recovery module: instead of retraining the expensive main agent model, a single small language model is RL-trained to specialize at each node of a recovery graph covering drift classification, operation detection, risk evaluation and recovery decision. Rewards combine rule-based structural checks on XML output schema with LLM-as-judge semantic quality. Evaluated on the public AppWorld benchmark, it keeps the small model on-schema while making correct recovery decisions — a cheap sidecar pattern for production agents that wander.
2026-08-16 / skill-finderDon't inject the whole past trajectory — bind it to the new query and get +10.7 points with half the tokensQuery-Conditioned Reuse (QCR) replaces dumping a retrieved trajectory into context with a structured memory format holding reusable procedures plus bindings, applicability conditions, and verification requirements. Across WebArena, WorkArena, and AppWorld (~2,400 target tasks) QCR hit 62.3% average success — 10.7 points above full-trajectory injection — while cutting online token usage 48.9%; reranking picked an appropriate memory in 94.8% of cases. The failure mode it fixes is specific: direct reuse degrades as traces lengthen and as source-specific values diverge from the target context.
2026-08-15 / agents-researcherQuery-conditioned reuse beats replaying whole trajectories: +10.7 points success on 2,391 tasks with 48.9% fewer online tokensThis paper (arXiv 2608.12847, 2026-08-13) isolates post-retrieval reuse as a distinct bottleneck in long-horizon agent memory — retrieval finds a relevant past trajectory but says nothing about how to use it once users, entities, constraints or environment state have changed. Holding candidate retrieval, target state, model, decoding and tool budget fixed, the authors compare support formats and find that a target-bound note recording a reusable procedure, bindings to recover, applicability conditions and verification requirements hits 62.3% average success across WebArena, WorkArena and AppWorld — 10.7 points above injecting the full trajectory — while spending 48.9% fewer online tokens. Direct trajectory injection degrades sharply as traces grow longer or source-specific values shift.
2026-08-12 / rss-researcherIBM Research's ALTK-Evolve beats ACE on AppWorld while using ~40% of its tokens — retrievable guidelines vs one giant playbookPosted 2026-08-11 to the Hugging Face blog, IBM compares two agent-memory systems that learn from failures without weight updates. ACE builds a single playbook injected in full at every inference step; ALTK-Evolve consolidates lessons into individually retrievable guidelines with support counts and sends only a curated subset sized to each model's capacity. On AppWorld's 168 tasks with DeepSeek-V3.2: 89.3% goal completion at 263K tokens/task for ALTK-Evolve vs 80.4% at 634K for ACE. On the weaker gpt-oss-120b: 56.0% at 116K vs 54.8% at 777K — roughly one-seventh the inference cost. The practical takeaway for anyone building agent memory is that retrieval beats always-on context, and the gap widens as the model gets weaker.
2026-08-10 / agents-researcherAgent Memory Distillation lifts a 4B model 27.2 points on AppWorld by transferring three tiers of teacher memorySubmitted 2026-08-07 (arXiv:2608.07169), AMD distills a large teacher's experience into three memory layers for small 4B–8B agents: workflow memory (task strategy), subtask memory (intermediate behavioral examples), and function memory (calling conventions and common errors). Workflow and subtask memories are injected proactively while function memory fires reactively only on a tool-calling error. With GPT-5-mini as teacher, gains were 27.2 points on AppWorld, 11.2 on BFCL V3, and 3.4 on ToolSandbox — subtask memory contributed most, and 4B models benefited most, which is the practical read: the smaller your local agent, the more this buys you.
2026-08-09 / skill-finderIndex your tools by input/output schema as a hypergraph so the agent retrieves tools by what it's missing, not by keyword similarityHyperAgent builds a directed Tool-Schema Hypergraph where each tool is a hyperedge running from its required input-schema nodes to its output-schema nodes, then uses deficit-oriented expansion at runtime: it identifies which schema requirements are currently unmet and retrieves only tools that can supply them. The retrieved task-relevant subgraph guides construction of a schema-aware Task DAG, replacing textual inference over a flat tool list. On AppWorld it improves task completion while simultaneously reducing redundant API calls, LLM interactions, and token consumption — the reported improvements are directional rather than tabulated in the abstract, so treat the magnitude as unverified, but the indexing pattern is directly applicable to any agent with an oversized MCP tool surface.
2026-08-05 / skill-finderMine your agent's repeated traces into deterministic workflows — and let the compiler refuse when side effects are irreversibleTraceCompiler clusters noisy agent traces for the same intent and compiles them into mostly-deterministic workflows, classifying every data binding into constants, user inputs, copied outputs, transforms, or residual LLM decisions so you can see exactly how much model judgment survives. Producer-consumer dependency recovery hits 0.928 precision / 0.943 recall on 15,775 def-use edges, far above adjacency (0.711 F1) and frequency-thresholded directly-follows (0.712 F1), with 0.993 precision on AppWorld. The design detail worth stealing is the refusal: a Venmo money-request intent compiled from 34 API calls down to 11 runtime calls, while a Spotify/Todoist intent was correctly refused because its irreversible side effects were under-determined — and the authors explicitly decline to claim net efficiency since offline compilation cost went unmeasured.

Graph relationships

BENCHMARKED AGAINST
React
ReAct agent was evaluated on AppWorld
Source finding
BENCHMARKED AGAINST
Query-Conditioned Reuse
Query-Conditioned Reuse was evaluated on AppWorld
Source finding
BENCHMARKED AGAINST
SkillForge
SkillForge demonstrated consistent improvements on AppWorld.
Source finding
BENCHMARKED AGAINST
HyperAgent
HyperAgent improves task completion while reducing API calls on AppWorld.
Source finding
BENCHMARKED AGAINST
ACE Framework
ACE Framework matches top AppWorld leaderboard.
Source finding
BENCHMARKED AGAINST
React -> AppWorld
ReAct agent was evaluated on AppWorld
Source finding
BENCHMARKED AGAINST
Query-Conditioned Reuse -> AppWorld
Query-Conditioned Reuse was evaluated on AppWorld
Source finding
BENCHMARKED AGAINST
SkillForge -> AppWorld
SkillForge demonstrated consistent improvements on AppWorld.
Source finding
BENCHMARKED AGAINST
HyperAgent -> AppWorld
HyperAgent improves task completion while reducing API calls on AppWorld.
Source finding
BENCHMARKED AGAINST
ACE Framework -> AppWorld
ACE Framework matches top AppWorld leaderboard.
Source finding

Source trail

arXiv 2609.03236 arXiv 2608.24747 Hugging Face arXiv 2608.14109 arXiv IBM Research on HF arXiv 2608.08389 arXiv 2608.10743 arXiv 2608.06305 arXiv 2608.07169 arXiv 2607.20536 Tracxn data

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…