27,000 AI Carb-Counting Queries Reveal Dangerous Variance — Gemini 2.5 Pro Estimates Ranged 55g to 484g for Same Photo

Diabettech / Hacker News·high signal

A diabetic researcher submitted 13 food photos 500 times each to GPT-5.4, Claude Sonnet 4.6, Gemini 2.5 Pro, and Gemini 3.1 Pro across 26,904 total queries. Claude Sonnet 4.6 performed best with 2.4% median variation and 0.9U insulin swing, while Gemini 2.5 Pro showed 11% variation with a worst-case 42.9U insulin swing — a potentially lethal dosing error. For a single paella photo, Gemini 2.5 Pro's estimates ranged from 55g to 484g of carbs. 238 points, 297 comments on HN.