A safety classifier hitches a ride on speculative decoding

Researchers turned speculative decoding's leftover machinery into a safety classifier that beats GPT-5.4-mini's zero-shot judgment, according to a new paper called Speculative Probing. That swaps a separate safety model for something the inference stack already runs. On multilingual prompt safety, the probes matched or beat two guard models built for the job, Qwen3Guard-Gen-8B and Llama-Guard-3-8B, without running either one.

The method appends a trained soft prompt to the end of the sequence. That turns the speculative-decoding module into a classifier instead of a token predictor. Because the KV cache is already resident during speculative decoding, running the classifier adds close to no extra compute.

The same pattern held across four tasks and four models total. Anyone running inference at scale with a draft model already in the loop has reason to check the benchmark tables before assuming this generalizes. The paper doesn't say how the probes hold up against prompts built to evade a soft-prompt classifier specifically. It also doesn't say whether the approach works on models that skip speculative decoding entirely.