ARC-AGI-3 Technical Report Alleges Gemini 3 and Other Frontier Models Memorized ARC-AGI-1/2 Benchmark Data

r/singularity / ARC Prize Foundation·medium signal

The ARC-AGI-3 technical report presents evidence that frontier models may have memorized earlier benchmark versions — Gemini 3's reasoning chain correctly referenced the integer-to-color mapping used in ARC-AGI tasks without being told what it was, strongly suggesting training data contamination. The ARC Prize Foundation says memorization is occurring in ARC-AGI-1 and ARC-AGI-2 'either incidentally or intentionally.' ARC-AGI-3 addresses this by keeping 110/135 environments private and requiring interactive skill acquisition rather than static pattern matching.