Fetching from the wire…
Public story · 2026-08-31 · high
The fix cuts a device-to-host round trip per decode round, and the commit lists an AI model as co-author alongside a human contributor.
Why now: Covered in llama.cpp's August 31 release notes.
llama.cpp folded its DFlash speculative encoder into the decoder's embedding branch, so one llama_decode call now handles both steps. Speculative decoding exists to skip redundant passes through the model, and forcing a device round trip on every round worked against that.
Before this change, the encoder ran as its own llama_encode call. It's a small op, just a fully-connected layer plus a norm. But running it separately forced a round trip from device back to host before the injection decode could re-upload the result. It also meant building a second computation graph every round.
Merging the two removes that round trip and the extra graph.
Claude Fable is listed as co-author on the commit, alongside a human contributor, per llama.cpp's b10715 release notes. This is a patch to how tensors move between decode calls, credited jointly to a human and an AI model. The notes don't say how the work split between the two, or what review it went through before merging.
For anyone running speculative decoding setups on llama.cpp, the result is fewer device syncs per round. That should show up as lower latency on anything using DFlash.
Each link below shares sources, entities, or timing with this story.
Barry Zhang and Mahesh Murag, the engineers who built Claude Skills at Anthropic, published a talk and engineering post that's gotten 14K+ likes and is reshaping how I think about agent development. The core argument: most agent approaches fail because they lack domain experti...
For about a year, "run your agent locally" meant accepting a model that couldn't reliably call a tool twice in a row. That excuse is gone. Meta Superintelligence Labs published Muse Glimmer today: a 29.6B dense causal transformer, 52 layers, 6,656 hidden dim, with a ~1.8B ViT-...
Huang used his inaugural X post on July 24 to publish "Open Weights and American AI Leadership," a three-page letter on Nvidia's own servers signed by 25 companies including Meta, Microsoft, IBM, Mistral, Mozilla, Hugging Face, a16z, Palantir and the Linux Foundation. Within a...
Moonshot AI released Kimi K3, a sparse mixture-of-experts activating 16 of 896 experts per token. That's about 1.8% of the pool live at any moment, with a 1M-token context window and native vision. Two new architectural pieces show up: Kimi Delta Attention and Attention Residu...
Jack Clark's Import AI 464 (around July 6) led with something I've been turning over all week. Claude Fable autonomously wrote what Clark calls "the first genuine (and fastest) megakernel" submitted to the KernelBench-Mega leaderboard. An 18.71x speedup in hand-written CUDA on...
Limit scope per step, use a strict plan/act/observe/refine cycle, and keep tools small. For classification, extraction, and routing tasks, local models match cloud performance at zero per-call cost.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.