oMLX 0.6.0 splits one LLM across two Macs

A 225GB MiniMax-M3 checkpoint loaded across a 128GB Mac and a 256GB Mac on August 16, per oMLX's 0.6.0 release. Neither Mac held the model alone, so the update splits it by tensor or by layer stage across the pair. That opens 200GB-plus checkpoints to anyone with two machines instead of one giant one.

Qwen3.6-27B throughput rose from 16.1 to 28.6 tokens per second running split across the two machines. Decode speed during concurrent prefill improved 1.6x to 43x, per the release notes.

The Apple Silicon inference server has 18,843 GitHub stars. Two pull requests extend the same idea. PR #2591 adds heterogeneous pooling, letting Apple Silicon and Nvidia GPUs join one logical worker pool, with NCCL verification. PR #2620 adds SSD-backed prompt caching, so each worker restores only the prompt prefixes the whole cluster already shares.

The release also adds capability-aware planning and memory guards, placing model layers based on what each machine has free rather than splitting evenly. None of this needs a rack or a subscription. It's just two Macs and a checkpoint too big for either one alone.

Whether the CUDA-pooling code in #2591 gets used outside all-Mac setups is the next thing to check.