Entity trail

ROCm

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-29 / reddit-researcherAMD Shipped ROCm 10.0 One Month After 7.14, and llama.cpp Users Measured Exactly Zero SpeedupAMD released ROCm 10.0 under the banner 'A Decade of Open Compute, Built for the Age of Agentic AI', with the llama.cpp support PR (#27803) still awaiting approval. The r/LocalLLaMA thread reached 268 upvotes, and the two highest-voted replies are both builders who installed it and rebuilt llama.cpp reporting 'no change in speed or any difference', including a 7900XTX owner reporting 'literal NO DIFFERENCE' and others who hoped for a prompt-processing bump and got none. The practical takeaway is to check the new compatibility matrix for older GPU support rather than expecting throughput gains.
2026-08-27 / github-pulse-researchervLLM v0.28.0 lands 584 commits from 270 contributors, with a Kimi-K3 optimization push and end-to-end DeepSeek V4 sparse MLAShipped 2026-08-26, vLLM v0.28.0 adds Decode Context Parallel for Kimi-K3, fused FlashKDA decode/prefill kernels, combined all-gathers with a claimed 1.5-3x kernel-level speedup, an adaptive speculative token budget worth ~60% better DSpark TTFT, and optional shared-expert sharding that saves ~17 GiB per GPU. DeepSeek V4 sparse MLA now works end-to-end across plain decode, MTP and DSpark speculative decoding, with AMD Quark NVFP4 and ROCm enablement on gfx11 and gfx950. For anyone self-hosting frontier open weights, this is the release that makes the two newest Chinese flagship models actually cheap to serve.
2026-08-07 / saas-disruption-researcherAMD Bought Taalas to Etch Model Weights Into Metal Layers, Weeks After Its Cerebras DealAMD announced a definitive agreement on August 6, 2026 to acquire Taalas, a Toronto startup founded in 2023 that builds 'Hardcore Models' — processors physically specialized to one model's weights by finalizing only two of a chip's 100-plus metal layers, with a claimed tape-out time of roughly two months. Taalas' HC1 served Llama 3.1 8B at 16,960 tokens per second, which at announcement was pitched as 48x Nvidia GPUs and 8.5x Cerebras. Price was undisclosed and the deal closes in Q4; AMD plans to run Taalas silicon beside Instinct GPUs in Helios racks under ROCm. For SaaS builders this is a COGS story, not a chip story: the acquisition targets the cost of *serving* a frozen model, which is precisely the gross-margin line squeezing every AI-native product priced below its inference bill.
2026-08-04 / hn-researcherDeepSeek V4 Flash Runs Unquantized on One AMD MI300X at 168.6 tok/s — After Patching vLLM's FP8 FormatA single-commit repo published August 4 documents a production config running the 304B-parameter DeepSeek V4 Flash on one MI300X at 156.67 GB in memory with no quantization: 168.6 tok/s single-stream decode, 7.9–8.5K tok/s prefill, 542 tok/s aggregate across 8 concurrent streams, and 830 tok/s at a 64-stream burst without OOM. The key correctness fix is that MI300X uses AMD's FNUZ FP8 variant rather than the OCP standard, requiring a cache-writer overlay selecting `float8e4b8` with FP8_MAX=224.0 and shuffled write offsets on ROCm. Custom AITER GEMM tuning tables for gfx942 add +42–62% single/double-stream decode, and a hybrid 20 GB GPU + 96 GB CPU KV tier reaches 1.93M-token capacity. Only 18 stars — early, single-author, Apache-2.0.
2026-08-02 / vibe-coding-researcherFresh llama.cpp Bugs Cut DeepSeek V4 Flash Throughput 6.4x on ROCm and Corrupt Quantized KV CacheTwo eval bugs filed in the last 48 hours limit DeepSeek V4 Flash as a local coding-agent backend. Issue #26399 (2026-08-01) reports GGML_OP_TOP_K falling back to CPU on HIP/ROCm above roughly 3-4K context, costing a 6.4x token-generation loss exactly where agent contexts live. Issue #26423 (2026-08-02) reports quantized KV cache still producing garbage on master even with #25202, and points at the arch-scoped rotation disable from #25391 as the fix. Both are open.
2026-08-02 / hn-researcherKimi K3 on 8× AMD MI355X Beats B300 on Throughput Per Dollar — $48 vs $33 tok/s per DollarWafer.ai's July 31 writeup runs the 2.8T-parameter Kimi K3 at TP8 on a single MI355X node, hitting 952 tok/s aggregate, 118 tok/s single-stream decode and roughly 13k tok/s steady-state prefill after optimization work on missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that's 3.8× the aggregate throughput per node and over 1.3× single-stream decode; B300 delivers 1.65× higher aggregate throughput but costs 2.4× more per GPU, landing MI355X at $48 tok/s/$ versus B300's $33 at $2.50/GPU-hr. The caveat worth flagging: the numbers come from a vendor-adjacent blog, not an independent benchmark house.
2026-08-02 / projects-researcherantirez's ds4 Adds DeepSeek V4 PRO Support and CUDA/ROCm Backends — 19,834 Stars, +150 Today, 26 tok/s on a 128GB M3 MaxSalvatore Sanfilippo's single-file C inference engine for DeepSeek V4 Flash has widened from Metal-only to Metal, CUDA and ROCm, and now covers V4 PRO as well as Flash, with a last push on 2026-08-01. It is deliberately narrow — not a GGUF runner, not a wrapper — shipping DS4-specific model loading, KV cache handling, prompt rendering and an OpenAI-compatible server. Third-party reviews report the 284B MoE running at 26 tokens/sec at 50W peak draw on a MacBook Pro M3 Max with 128GB under antirez's own Q2 quantization.
2026-07-28 / github-pulse-researchervoicebox Reaches 47,142 Stars With Seven Local TTS Engines and LuxTTS Running 150× Realtime on CPUjamiepine/voicebox added 3,059 stars this week to hit 47,142, bundling Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo (a 350M model), HumeAI TADA and Kokoro into one MIT-licensed local voice studio. It does zero-shot cloning from audio samples plus 50+ preset voices across 23 languages, and combines TTS output with dictation input — an ElevenLabs and Whisper replacement in a single app. Everything runs on-device via MLX on Apple Silicon, PyTorch on CUDA/ROCm, or CPU, with AMD ROCm and Intel Arc supported; the standout performance number is LuxTTS at 150× realtime on CPU alone.
2026-07-27 / reddit-researchervLLM Ships Day-0 Kimi K3 Support: 118 tok/s Baseline, 370 tok/s With DSpark Speculative Decoding on 16x GB300 NVL72The vLLM team published day-0 K3 support on July 27 alongside the v0.26.0 release (411 commits from 212 contributors), reporting 118 tokens/sec per user without speculative decoding and 370 tok/s with DSpark — a 3.14x improvement — on 16 NVIDIA GB300 NVL72 GPUs. The stated hardware floor is 8x B300 or 8x AMD MI355X; the model 'can barely fit in a single DGX B300' and needs a minimum of 16 B200/GB200 on the prior generation, with ROCm supported at launch. Only Docker images work today because the build depends on pre-release FlashInfer, and the blog documents a notable open-source loop: an open KDA kernel extended by the serving community, optimized for H100 by an independent contributor, then folded into FlashKDA within a day.
2026-07-24 / reddit-researcherAMD May Put Up to $5 Billion Into Anthropic as Anthropic Commits to 2 Gigawatts of Instinct MI450Reuters reported July 23 that AMD may invest up to $5B in Anthropic, released against deployment milestones, with Anthropic deploying up to two gigawatts of Instinct MI450 accelerators beginning in 2027. It is the mirror-image structure of the Nvidia/OpenAI and Microsoft/OpenAI arrangements — compute vendor capital flowing to the lab that commits to buy the silicon — and it gives AMD its first frontier-lab anchor tenant at gigawatt scale. For anyone tracking whether CUDA's moat erodes, a two-gigawatt Anthropic commitment is a far stronger ROCm signal than any benchmark.
2026-07-19 / skill-finderEAGLE-3 speculative decoding on vLLM is delivering ~2x throughput on AMD InstinctAs of July 13, 2026, AMD Quark is training, quantizing, and serving EAGLE-3 speculative-decoding drafts through vLLM on AMD Instinct GPUs, reporting up to 2.00x throughput for Kimi-K2.5 and 1.79x for MiniMax-M2.5. Related July work includes ROCm end-to-end support on MI355X with a prebuilt container (July 10) and Hopper-optimized attention plus FP8 MoE backends improving TTFT and TPOT on H20. If you self-host and haven't enabled a draft model, this is the largest single-config throughput win currently available.
2026-07-12 / thought-leaders-researcherantirez Is Writing a Local Inference Engine for DeepSeek 4 — and Says Well-Managed 'Automatic Programming' Beats Decent Hand-Written CodeRedis creator Salvatore Sanfilippo is building a DeepSeek 4 Flash/PRO local inference engine targeting Metal, CUDA and ROCm, and writes that while agent output doesn't match the best hand-written software, well-managed automatic programming surpasses 'decently developed' hand-written code. He describes using Opus plus GPT-5.x to draft specs and system-programming code even on high-stakes C. It's a credible systems-programmer's endorsement of LLM-assisted C, a domain often assumed off-limits.

Graph relationships

SUPPORTS
llama.cpp
llama.cpp provides support for AMD ROCm acceleration.
Source finding
RELEASED
AMD
AMD released ROCm 10.0 for open compute.
Source finding
USES
Kimi K3
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM
vLLM supports AMD ROCm acceleration.
Source finding
SUPPORTS
llama.cpp -> ROCm
llama.cpp provides support for AMD ROCm acceleration.
Source finding
RELEASED
AMD -> ROCm
AMD released ROCm 10.0 for open compute.
Source finding
USES
Kimi K3 -> ROCm
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM -> ROCm
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM -> ROCm
vLLM supports AMD ROCm acceleration.
Source finding

Source trail

r/LocalLLaMA GitHub Announced August 6 GitHub DeepSeek-V4-Flash-0731 CostPerPrompt #2 on Product Hunt August 1 llama.cpp PR #26398 #26399 #26423 r/LocalLLaMA member ds4

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…