Tip: Get 24+ tok/s from 30B MoE Models on a $200 GTX 1080 Machine Using RotorQuant KV Cache Quantization with 128K Context

r/LocalLLaMA·medium signal

Practitioner reports running Qwen 3.6 35B-A3B and Gemma 4 26B-A4B at 24+ tok/s on a secondhand machine (i7-6700, GTX 1080 8GB VRAM, 32GB RAM) using llama.cpp with TurboQuant/RotorQuant KV cache quantization enabling 128K context windows. The combination of sparse MoE architectures (only 3-4B active parameters) with aggressive KV cache quantization makes 30B+ models viable on 8-year-old hardware. Total cost: approximately $200 for the complete secondhand build.