Tip: plugin eval silently drops `tool_used: Skill` graders from the score to keep the two arms comparable

Claude Code Docs·medium signalXBlueskyLinkedInCopy link

A grader that checks the skill was invoked can never pass in the without-arm, so counting it would push that arm toward zero and inflate the delta. Claude Code reports such graders with `scored: false` in both arms and shows them in the with-arm as pass/fail indicators only, along with anything you mark `arm: with-only`. Set `arm: both` when you want the opposite, such as a must-not-invoke check with `min: 0` and `max: 0`; under `--ablation none` nothing is excluded, so the same suite yields a different absolute score in the two modes.