vLLM's MFU accounting hardcoded activations at 2 bytes while sizing weights from the real dtype

GitHub·low signalXBlueskyLinkedInCopy link

PR #57070, merged 2026-09-17T00:23Z, fixes `BaseConfigParser` pinning `activation_byte_size = 2` on the line after it resolved `weight_byte_size` from the model's own `torch_dtype`, so the two disagreed. A model served with `--dtype float32` had every activation term in its MBU accounting counted at half its real size while weight terms stayed correct. bfloat16 and float16 deployments are unchanged; float32 goes from 2 to 4 bytes, doubling reported activation read and write traffic and dropping reported MBU, which the author notes is the correction rather than a regression.