Fetching from the wire…
Infra2026-08-09 · source-backed
DCP splits KV cache across GPUs by token position rather than attention head, eliminating the duplication that caps tensor parallelism on long contexts (vLLM). On 8×B200 with Kimi K2.6, baseline TP plateaued at ~1,863 tok/s/GPU at concurrency 64 while DCP reached 6,091 tok/s/GPU at concurrency 512 using only 82% of KV memory, staying stable past 200K-token sequences. Enable with --tensor-parallel-size N --decode-context-parallel-size M. Helps MLA/GQA models on 64K+ contexts with large-input/short-output agentic traffic, which describes basically every coding agent. Hurts on short contexts or weak GPU interconnect.
Each link below shares sources, entities, or timing with this story.
The open-weight race just changed constraint. Moonshot AI suspended all new consumer subscriptions on July 20, roughly 48 hours after Kimi K3 launched, because request volume pushed its compute cluster to capacity. Remaining GPUs are reserved for existing paid subscribers. Tec...
The assumption that proprietary models own the coding benchmark crown just broke. Moonshot AI's Kimi K2.6 leads on 5 of 8 major agentic coding benchmarks while being the only open-weight model in the top tier. SWE-Bench Pro: 58.6% vs GPT-5.4's 57.7% and Claude Opus 4.6's 53.4%...
Moonshot AI dropped Kimi K2.6 today and the numbers are hard to ignore. One trillion parameters total, 32 billion active per token across 384 experts, 256K context window, and native multimodal input. It scores 58.6 on SWE-Bench Pro versus GPT-5.4's 57.7 and Claude Opus 4.6's...
Wafer.ai ran K3 at TP8 on a single MI355X node: 952 tok/s aggregate, 118 tok/s single-stream decode, ~13k tok/s steady-state prefill after fixing missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that'...
Cloudflare added Moonshot AI's Kimi K2.5 to Workers AI on March 19, making it the first frontier-scale open-source model available on edge compute with a full 256K context window, multi-turn tool calling, vision inputs, and structured outputs (Cloudflare Blog). Cloudflare repo...
PR #19378 landed in llama.cpp this week, and I think most people are underselling what it means. Backend-agnostic tensor parallelism via --split-mode tensor makes multi-GPU inference work across AMD, Intel, and Apple Silicon. Not just CUDA. Everything. For context: llama.cpp h...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.