SafeSteer: Localized On-Policy Distillation Cuts the Alignment Tax for LLM Safety

arXiv·medium signal

Introduces SafeSteer, which aligns LLMs for safety without the typical capability degradation known as the 'alignment tax.' Uses localized on-policy distillation to preserve general capabilities while enforcing safety constraints, avoiding reliance on massive general-purpose data or auxiliary models. Practitioners fine-tuning models for safety can adopt this to avoid the painful tradeoff between safety compliance and benchmark performance.