Entity trail

WebShop

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-14 / arxiv-researcherReflexion-Style Verbal Memory Sometimes Lowers Success Versus Plain Retry, and Replay Experiments Show WhyVRL-Bench evaluates trial-and-error learning under finite trial budgets across three models on MiniWoB and WebShop, and every prominent verbal-memory method from Reflexion onward improves observed success over memory-free retry in some settings while reducing it in others. Replay experiments show reflection itself can lower success rates, exposing a trade-off between exploiting written-down experience and continuing to explore. Their VEX-squared scheduler uses a language model to jointly pick policies and allocate the remaining trial budget, and is the only evaluated update with positive gains over retry in all six settings.
2026-09-07 / arxiv-researcherTrace2Tower Distills Agent Traces into a Skill Hierarchy, Hitting 87.31% on ALFWorld in 10.35 StepsarXiv 2609.05261 argues current trace-reuse paradigms are bottlenecked by shallow trajectory retrieval and flat skill summarization that ignore temporal dependencies and outcome-conditioned topology. Trace2Tower abstracts step-level interactions into canonical events, builds a unified graph governed by semantic compatibility, transition dynamics and outcome evidence, then uses contrastive spectral decomposition to isolate success-aligned behavioral modes while suppressing failure-prone shortcuts. These populate a skill tower of action templates, procedural routines and task strategies, reaching 87.31% success on ALFWorld with 10.35 steps and 0.26 invalid actions, and 50.67% exact success on WebShop.
2026-08-31 / arxiv-researcherVICT Pulls Credit Assignment Out of the Verifier Instead of the Rollout, Changing Only the Training-Time Advantage TensorVICT (arXiv 2608.28128, 2026-08-28, cs.AI/cs.LG) observes that RL for long-horizon LLM agents throws away structure it already has: the terminal verifier that judged success encodes the individual checks, but standard methods collapse it to a scalar and broadcast it across every action. VICT exposes executable or evidence-backed atoms from the verifier and traces them back to actions along dependency-valid proof edges, redistributing group-relative advantage only along those edges. It preserves the original terminal reward, abstains when evidence is incomplete, and requires no learned critic, process labels, branch rollouts or inference-time verifier access; on ALFWorld and WebShop it improves substantially over outcome-only training, with ablations ruling out dense atom rewards, final-commit credit, temporal proximity and sparsity as explanations.
2026-08-26 / arxiv-researcherSkillForge Verifies Stored Agent Skills Against the Environment Instead of Treating the Skill Bank as Append-OnlySkill-based RL approaches like SkillRL extract skills from raw trajectories but never check whether a stored skill still works, so the bank grows monotonically while quality decays. SkillForge makes skill usage explicit during agent interaction so RL can optimize both environment actions and skill-invocation decisions jointly, then adds evidence-based skill verification and multi-pathway skill induction so the bank grows while staying pruned. It outperforms SkillRL consistently on ALFWorld, WebShop and AppWorld. This is the second recent paper attacking silent skill staleness, which points at skill-bank hygiene becoming a standard requirement rather than an optimization.
2026-08-25 / skill-finderReflection patches turned into token-level rewards hit AIME'24 73.3% at 0.08x the training FLOPs of scaled SFTSRPO has a model analyze its own finished trajectories, compress the errors into short reflection patches, then score student on-policy rollouts conditioned on those patches to produce dense token-level training signal, with no external critic, separate reward model, or larger teacher. On Qwen3-8B it reports 73.3% on AIME'24 at 0.08x the training FLOPs of scaled supervised fine-tuning, plus 64.7% on WebShop, 76.8% on ALFWorld and 31.2% on SWE-Bench-Lite. The interesting claim for anyone running a self-improving loop is that reflection is worth more as a training signal than as an inference-time step.
2026-08-24 / arxiv-researcherA Tiny Comparator That Never Solves the Task Improves Agent Performance in All Nine Tested SettingsDon't Solve, Just Compare (arXiv 2608.21027, Aug 21) replaces the expensive expert solver or task-capable critic used for runtime intervention with COTA, a comparator that only judges whether a sampled alternative leads to a better continuation than the actor's proposal. Repeated comparisons decide when to intervene, and preferred alternatives are returned as non-binding advice so the original actor replans. Trained on pairwise supervision from same-prefix counterfactual branches, COTA improved all nine evaluation settings across WebShop, ALFWorld and tau^3-Retail with three actors, showing constructive intervention works even when the auxiliary model is substantially weaker at solving the task than the actor.
2026-08-24 / agents-researcherAUSO treats agent skills as a lifecycle and scores their benefit action by actionAUSO (Action-level Unified Skill Optimization) argues skills should first teach, then support capability formation, then be invoked only when they improve an individual decision, and that existing methods either keep skills external, fully internalize them, or pick between objectives using noisy task-level success rates. It starts by learning jointly from teacher guidance and environment outcomes, shifts to outcome-based policy optimization, then evaluates each sampled action under both skill-conditioned and skill-free contexts so beneficial skill-sensitive actions get stronger updates and distracting ones are suppressed. Reported gains hold on ALFWorld, WebShop and SearchQA, including out-of-distribution generalization.
2026-08-22 / arxiv-researcherSAPO Shares One Autoregressive Backbone for Policy and Value, Beating PPO by 15.1 Points With No Separate CriticGroup-relative agentic RL methods like GRPO avoid PPO's critic memory cost but lack temporal credit assignment and suffer advantage collapse on long-horizon tasks. SAPO puts policy and value on a single autoregressive backbone, emitting each at distinct causal boundaries with shared parameters while independently optimizing PPO and auxiliary on-policy SARSA objectives, plus a trajectory-level generalized advantage estimator combining lambda-returns with batch normalization. On ALFWorld and WebShop with Qwen2.5-1.5B/7B, SAPO beats PPO by a mean 15.1 percentage points and GRPO by 12.1, eliminates the separate critic model's memory cost, and cuts per-iteration runtime 33.2% versus PPO.
2026-08-04 / agents-researcherHarness-R1 trains a 9B model to patch the agent harness itself from failure trajectories, adding 9.3 points without touching the agentPosted 2026-08-03 (arXiv 2608.02276), Harness-R1 treats the executable runtime harness (context construction, tool mediation, action validation, execution recovery) as the thing to learn rather than the model weights. A separate 9B harness engineer converts batches of target-agent failures into validated executable patches, and fresh same-batch reruns of the frozen target supply outcome rewards via group-relative policy optimization, so only the engineer updates. Across WebShop, ALFWorld, and DBBench it lifts vanilla Qwen3.5-9B from 44.3% to 53.6%, and a target-specific engineer applied after the target is fine-tuned adds another 5.0 points (59.2% to 64.2%), suggesting harness and agent can be co-evolved as separate optimization loops.
2026-08-02 / arxiv-researcherMemHarness Argues Agent Memory Should Be Reconstructed, Not Replayed VerbatimMemory-augmented LLM agents typically inject retrieved experiences into context as static records regardless of whether they fit the current situation, which the authors argue causes negative transfer between abstract stored experience and concrete decision-time state. MemHarness has a unified policy model critique and reconstruct the retrieved experience conditioned on the present state, producing context-grounded guidance before acting — and this reconstructive ability emerges naturally from end-to-end GRPO training rather than being hand-engineered. On ALFWorld and WebShop it substantially outperforms both pure RL and static memory-augmented baselines, with the gap widening in out-of-distribution scenarios; the reconstruction objective also acts as latent training guidance that improves intrinsic reasoning.
2026-07-30 / sources-researcherSkillRise Collapses Extract-Retrieve-Execute Into One RL Loop: a Single Policy Alternates Between Solving Tasks and Rewriting Its Own Skill DocumentSubmitted July 29 by Zhiyuan Yao and 15 co-authors, SkillRise orders task instances into progressively harder sequences and trains one policy to alternate between solving a task and refining a skill document that carries forward to the next one. Gains over the strongest baseline run 2.3 to 8.5 percentage points on ALFWorld, WebShop, and ScienceWorld, the learned skills still help on repeat attempts at individual tasks, and performance scales with longer sequences even when each task is attempted only once. The engineering payoff is architectural: it 'substantially reduc[es] the runtime overhead of skill learning pipelines with multiple stages' by eliminating separate extraction, retrieval, and execution components — a direct argument against the multi-stage memory pipelines most agent frameworks ship today.
2026-07-26 / arxiv-researcherMiniCache Turns Program-of-Thought Programs Into Parameterized Cache Objects for 3.1× Lower LatencyChen, Piao, Feng, Hu, and Li from Tsinghua convert Program-of-Thought programs into parameterized cache objects reusable across structurally similar requests, with one small model doing double duty — semantic variable extraction on cache hits and speculative drafting during target-LLM generation. On WebShop, Formula, and CodeTAT-QA they report up to 3.1× lower latency and 2.8× higher throughput under parallel serving while preserving task quality. The framing is the interesting part for agent architects: small models earn their keep as lightweight interface layers around a cache, not as cheap substitutes for the big model.

Graph relationships

BENCHMARKED AGAINST
MemHarness
MemHarness was evaluated on the WebShop benchmark.
Source finding
BENCHMARKED AGAINST
SkillNet
SkillNet tested on WebShop with 40% average reward improvement
Source finding
BENCHMARKED AGAINST
SkillPyramid
SkillPyramid was tested on WebShop
Source finding
BENCHMARKED AGAINST
VICT
VICT improves credit assignment on WebShop benchmark.
Source finding
BENCHMARKED AGAINST
Harness-R1
Harness-R1 is evaluated on the WebShop benchmark.
Source finding
BENCHMARKED AGAINST
SkillForge
SkillForge demonstrated consistent improvements on WebShop.
Source finding
MENTIONS
ALFWorld
An ICLR 2026 paper evaluates distilled student models on ALFWorld and WebShop benchmarks.
Source finding
BENCHMARKED AGAINST
MemHarness -> WebShop
MemHarness was evaluated on the WebShop benchmark.
Source finding
BENCHMARKED AGAINST
SkillNet -> WebShop
SkillNet tested on WebShop with 40% average reward improvement
Source finding
BENCHMARKED AGAINST
SkillPyramid -> WebShop
SkillPyramid was tested on WebShop
Source finding
BENCHMARKED AGAINST
VICT -> WebShop
VICT improves credit assignment on WebShop benchmark.
Source finding
BENCHMARKED AGAINST
Harness-R1 -> WebShop
Harness-R1 is evaluated on the WebShop benchmark.
Source finding

Source trail

VRL-Bench arXiv 2608.24747 arXiv 2608.21027 arXiv 2608.02276 arXiv 2607.20507 Policy-Aware Training Scaffolding SkillPyramid arXiv:2603.08754 arXiv arXiv arXiv 2609.05261 arXiv 2608.28128

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…