Benchmark finds 70% unsafe execution in Codex, Claude Code

SkillLogic's SLBench benchmark clocked unsafe execution rates up to 70% across six LLM backbones, including Claude Code and Codex, per the SkillLogic paper. The tool checks skill files for eight logical relations, things like preconditions, constraints, and fallbacks. Then it runs 86 test cases to see whether the model actually honors them.

Anyone installing skills from the public ecosystem is exposed. The same researchers scanned more than 5,000 public skill files and found 70% contain at least one of these relations. In testing, a mishandled precondition led to privacy leaks and unsafe config changes, exactly the failure mode the benchmark is built to catch.

There's a partial fix. The team's own mitigation, SLGuard, cut violations by 63% when layered on top. That's real progress, but it still leaves over a third of the unsafe runs getting through. The paper doesn't say whether SLGuard scans skills already sitting in public registries or only catches new ones going forward. That's the detail I'd want before trusting the fix.

The failure rate isn't the real story here. It's that 70% of the skills already circulating carry this same unscanned logic. Most of the marketplaces distributing them have nothing like SLGuard checking files before they ship. I'd bet that number barely moves by the time the next wave of skill files lands. If you're pulling skills into Claude Code or Codex from a public registry, read the precondition logic yourself before you trust it.