OSS
Show HN: Open-Source Red-Team Playground for AI Agents — First Challenge Compromised in 60 Seconds
Fabraix is an open-source red-teaming platform for AI agents with live agents using real tools, published system prompts, and public exploit transcripts released after each challenge concludes. The first challenge (get the agent to call a forbidden tool) was solved in 60 seconds, and the key finding from HN discussion is that effective attacks don't hide intent — they reframe context to make dangerous actions look legitimate, making LLM-as-judge defenses fragile. The transparency model (public exploit transcripts) is what distinguishes this from internal security testing — real attack patterns rarely match internal predictions.
Source
↳ Follow the thread