A retrofit of DeepSeek V4.1 Flash's KV approximation cut Qwen3-8B prefill time in half
nowokay blog / huggingface.co/kishida/Q3-8B-KVA-Projector (via r/LocalLLaMA, 327 upvotes)·medium signal
Developer Naoki Kishida trained a small projector that writes the KV cache for Qwen3-8B's later layers during prefill, while the full model still handles decoding. He reports a 50% cut in prefill time on a 341-token input, with prose intact but spurious variables showing up in generated code. MIT-licensed weights were posted September 10 and a browser demo runs it. An r/LocalLLaMA user has started a port to Qwen3.8-27B for a single RTX 3090.