Benchmark ties higher safety scores to more refusals

A testing platform called aiXamine ran more than 120 large language models through 46 safety, security and privacy checks.

It found that tighter alignment makes models refuse more often. One model that scored 99.3 on safety alignment turned down one in three benign queries during testing.

A model's privacy score barely moved with its safety or security score in the same testing. The three risks don't rise or fall together, so a high score on one doesn't predict the others.

The paper treats safety, security and privacy as connected, not separate rankings. It ran the checks across nine services, in more than 5,000 test runs total.

One model was distilled off-policy, without a correction step back onto the teacher's own outputs. Its robustness score dropped from 56.9 to 2.6 on the same base architecture.

A related quantization study found a 4-bit compressed model beating its uncompressed teacher on most benchmarks. Compression trade-offs cut in different directions depending on the method.