Fine-Tuning Security Classifiers Creates Evasion Vulnerabilities Invisible to Standard Evaluation

arXiv 2606.27091·medium signal

LLMs fine-tuned for security classification are usually evaluated on held-out data from the same distribution as training — which can hide vulnerabilities introduced by fine-tuning itself. The study (using a Foundation-Sec model) shows models learn token-level indicator semantics that keep canonical accuracy high but fail under behavior-preserving transformations like PowerShell alias substitution, command reconstruction, string construction, execution indirection, and case mutation. It's a warning that standard eval splits overstate the robustness of fine-tuned detectors.