Tools
vLLM adds CPU FP8 W8A8 for dense and MoE, validated on DeepSeek-R1
PR #49942 (merged 2026-09-18 09:59 UTC) adds FP8 W8A8 linear and MoE support for Intel DMR CPUs via `CPUFP8W8A8ScaledMMLinearKernel`, `CPUFp8W8A8BlockScaledMMKernel` and `CPUExpertsFp8W8A8`. Validated on Llama-3.1-8B-Instruct-FP8-dynamic (per-channel weight, dynamic per-token activation), Qwen3.5-35B-A3B-FP8 and DeepSeek-R1 (MoE + MLA, block-wise 128x128), with 375 kernel tests passing and end-to-end gsm8k of 0.802 for Qwen3.5-35B-A3B-FP8 on DMR. FP8 MoE on CPU makes a quantized 35B-A3B a plausible CPU-only serving target.
Source
↳ Follow the thread