Item Response Theory across 8 safety benchmarks and 192 models cuts eval cost 97-99% and catches sandbagging.

arXiv 2608.05086, billed as the largest psychometric analysis of LLM safety evals to date, finds three interpretable factors (refusal strictness, truthfulness, contextual harm) explain most between-model variance. Psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, ~10 adaptively chosen items sufficing for several benchmarks. IRT also supports per-model audits that detect naive sandbagging and silent model swaps behind an API. Directly usable if you maintain an eval harness against a hosted endpoint.