Multiverse's quantized model beats the shrunk version, not the original

Multiverse Computing published a claim that a compressed 4-bit model beats its own full-precision original. Read closely, that's not what happened. The Quantization-Aware Healing post describes cutting a 120B model to 60B in BF16, then compressing that 60B model further to mxfp4. The mxfp4 version beats the 60B BF16 checkpoint. It doesn't beat the original 120B model.

That gap matters. "Compression beats the source" and "compression beats an already-shrunk checkpoint" are different claims. The first would change how people weigh precision against size in production. The second is a narrower result about healing a model after most of its parameters are gone.

r/LocalLLaMA caught the gap within hours of the post going up. The sharpest technical objection came from the author of Heretic, and it targeted the method itself. Multiverse uses KLD against the teacher model's output distribution as its loss function during healing. The Heretic author calls that reasonable, but argues first-token KLD poorly predicts how a model diverges across a full response. A model can match the teacher's first-token distribution and still drift over a full response.

Multiverse's post doesn't address that critique. If quantized models start getting pitched as upgrades over their full-size sources, this comparison is worth checking before you repeat it.