arXiv: 'ClawSafety: Safe LLMs, Unsafe Agents' — Benchmark Shows Safety Collapses When LLMs Become Agents

arXiv·medium signal

New paper (2604.01438) introduces a 120-scenario benchmark across software engineering, finance, healthcare, law, and DevOps showing that LLMs rated 'safe' in isolation exhibit substantially higher risk when deployed as autonomous agents with real tool access. The benchmark tests harm domains, attack vectors, and harmful action types in realistic high-privilege professional workspaces. Key finding: model-level safety evaluations are insufficient — agent-level evaluation frameworks are needed before deploying agentic systems in production.