Public story · 2026-03-04 · source-backed

AgentLAB — First Long-Horizon Agent Attack Benchmark

Issue

2026-03-04

Confidence

source-backed

Sources

Redaction

passed

Story

5 novel attack types (intent hijacking, tool chaining, task injection, objective drifting, memory poisoning) across 28 environments. Key finding: single-turn defenses fail against multi-turn adversarial strategies. (arXiv 2602.16901)

ToolHazard synthesizes adversarial agent environments instead of hand-building them.
arXiv 2608.11878 replaces the handful of manually implemented injection-testing environments with an Environment Simulator, Attacker Agent, and User Simulator that generate executable stateful environments and discover viable injection points automatically. Injection timing an...
A Hidden HTML Comment in a Pull Request Owns Your Review Agent, and Microsoft Hasn't Patched It
Someone opens a PR against your repo. The description looks normal in the browser. Buried in it is . Invisible in the Azure DevOps web UI. Fully visible to your review ag...
Multi-turn attacks crack AI agents 5-14% of the time
"Adaptive Adversaries" (arXiv:2607.18063) tests agents against attackers that adapt across turns instead of firing one-shot prompts. Claude Opus 4.6 and GPT-5.4 tied at 5.4% aggregate, but per-scenario variance was extreme, with Opus hitting 60% on one scenario where competito...
LLM Web Agents Fail Dark Patterns 41–72% of the Time
The first systematic study of deceptive UI impact on LLM web agents, accepted at IEEE S&P 2026, tested against real e-commerce, streaming, and news dark patterns. Gemini 2.5 Pro: 65.78% susceptibility. Claude 3.7 Sonnet: 53.79%. GPT-4o: 51.26%. Guardrail models and prompt post...
benchflow-ai/skillsbench — Agent Skills Benchmark (396 stars)
First standardized benchmark for AI agent skills. 86 tasks, 11 domains, 7,308 test trajectories. Critical finding: curated skills +16.2%, self-generated skills +0%. Run it against your own skills. GitHub | Paper
CamoDocs poisoning attack hides its trigger from RAG filters
Existing RAG poisoning is filterable because the adversarial chunk contains the query. CamoDocs chunks synthesized benign and adversarial drafts, swaps selected tokens in benign chunks for dispersion tokens that spread the poisoned embeddings, then coherence-filters for readab...

Source trail

arXiv 2602.16901 More from arxiv.org

Entities

AgentLAB Horizon Agent Attack Benchmark

Provenance

Canonical issue: Ramsay Research Agent — 2026-03-04
AI generated: no
Story unit: 2026-03-04-agentlab-first-long-horizon-agent-attack-benchmark
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

AgentLAB — First Long-Horizon Agent Attack Benchmark

Story

Related stories

ToolHazard synthesizes adversarial agent environments instead of hand-building them.

A Hidden HTML Comment in a Pull Request Owns Your Review Agent, and Microsoft Hasn't Patched It

Multi-turn attacks crack AI agents 5-14% of the time

LLM Web Agents Fail Dark Patterns 41–72% of the Time

benchflow-ai/skillsbench — Agent Skills Benchmark (396 stars)

CamoDocs poisoning attack hides its trigger from RAG filters

Source trail

Entities

Provenance

AgentLAB — First Long-Horizon Agent Attack Benchmark

Story

Related stories

ToolHazard synthesizes adversarial agent environments instead of hand-building them.

A Hidden HTML Comment in a Pull Request Owns Your Review Agent, and Microsoft Hasn't Patched It

Multi-turn attacks crack AI agents 5-14% of the time

LLM Web Agents Fail Dark Patterns 41–72% of the Time

benchflow-ai/skillsbench — Agent Skills Benchmark (396 stars)

CamoDocs poisoning attack hides its trigger from RAG filters

Source trail

Entities

Provenance

Story

Related stories

ToolHazard synthesizes adversarial agent environments instead of hand-building them.

A Hidden HTML Comment in a Pull Request Owns Your Review Agent, and Microsoft Hasn't Patched It

Multi-turn attacks crack AI agents 5-14% of the time

LLM Web Agents Fail Dark Patterns 41–72% of the Time

benchflow-ai/skillsbench — Agent Skills Benchmark (396 stars)

CamoDocs poisoning attack hides its trigger from RAG filters

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance

Story

Related stories

ToolHazard synthesizes adversarial agent environments instead of hand-building them.

A Hidden HTML Comment in a Pull Request Owns Your Review Agent, and Microsoft Hasn't Patched It

Multi-turn attacks crack AI agents 5-14% of the time

LLM Web Agents Fail Dark Patterns 41–72% of the Time

benchflow-ai/skillsbench — Agent Skills Benchmark (396 stars)

CamoDocs poisoning attack hides its trigger from RAG filters

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance