Apple LLM in a Flash Technique Enables Qwen3.5-397B Local Inference on 256GB Mac

simonwillison.net·medium signal

Developer Dan Woods applied Apple's 'LLM in a Flash' memory offloading research to run Qwen3.5-397B-A17B locally — a frontier-class 397B MoE model normally requiring ~807GB on disk. Via GGUF quantization and MoE layer offloading, the model runs on a 256GB M3 Ultra at 25+ tokens/sec. Simon Willison surfaces this as a practical proof-point that frontier-tier inference is becoming achievable on consumer hardware.