Sup AI Sets New SOTA on Humanity's Last Exam: 52.15% via Confidence-Weighted Multi-Model Ensemble — 7.4% Above Gemini 3 Pro

Sup AI / Hacker News·low signal

Sup AI achieved 52.15% accuracy on Humanity's Last Exam (HLE), a 2,500-question benchmark spanning advanced math, science, and reasoning, beating the previous best single model (Gemini 3 Pro Preview at 44.74%) by a statistically significant margin (p < 0.001). The system dynamically routes each question to frontier models best suited to the problem, then synthesizes answers weighted by confidence and inter-model agreement. Shows ensemble approaches still outperform individual frontier models on hard reasoning.