VLMs default to vision but let priors override it.

Niclas Lietzow, Danielle Bitterman, and Carsten Eickhoff probe what happens when a vision-language model's eyes disagree with its memorized world knowledge, identifying a "vision-default, prior-override" causal mechanism. This is directly useful for debugging the maddening class of VLM hallucination where the model describes what it expects instead of what's actually in the image. It gives you a mechanistic handle on when to trust perception versus baked-in memory.