Zvi: Astra scores 96.7% on ARC-AGI-3 with reasoning disabled, and that is the part that worries him

Don't Worry About the Vase (Zvi Mowshowitz)·high signalXBlueskyLinkedInCopy link

Zvi's Sep 12 post calls Astra a bigger jump than Fable 5 to Fable 5.1, citing 98% on FrontierMath Tier 4, 98.1% on extended NYT Connections against Fable 5.1's 90%, the first autonomous Montezuma's Revenge clear, and a one-shot Portal completion. The monitorability problem is Neel Nanda's finding that Astra scores 159 on Epoch ECI with no chain-of-thought, four points off Fable 5.1's full-reasoning 163 and well above Fable's ~128 no-CoT score. Zvi calls the 100% ExploitBench figure a chart crime likely reflecting contamination, and his practical advice is to run both Fable 5.1 and Astra on hard questions rather than switching.