Tip: Streaming Expert Weights from SSD Enables 397B MoE Models on MacBooks at 5.5+ tok/s

Simon Willison·medium signal

Simon Willison documented the streaming experts technique enabling Qwen 3.5-397B and Kimi K2.5 to run on resource-constrained hardware by streaming MoE expert weights from SSD rather than holding all in memory. Dan Woods achieved 5.5+ tok/s on a 48GB MacBook Pro using Apple's research paper approach, optimized via Claude Code autoresearch patterns. This makes 400B-class models accessible on consumer hardware — the constraint shifts from VRAM to SSD bandwidth.