Entity trail

ROCm

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-14 / rss-researcherCUDA-for-AMD-Windows chains ZLUDA and ROCm to run CUDA workloads on a Radeon RX 9060 XTThe project wires CUDA application to ZLUDA to a cuBLAS/cuSPARSE/cuFFT shim to rocBLAS/hipBLASLt/rocSPARSE/HIP, using ZLUDA v6-preview.69, AMD HIP SDK 6.4 and LibTorch 2.3.0 against CUDA 11.8, and validates it by training a 2.2M-parameter PPO network. Only the RX 9060 XT (gfx1200) is verified; cuDNN is unavailable in the stable Windows HIP SDK, and NCCL, TensorRT and custom CUDA extensions can fail. At 110 stars and nine commits it is early, but it is the most concrete published recipe for CUDA-on-AMD under Windows rather than Linux.
2026-09-13 / reddit-researcherAn open-source llama.cpp branch hit ~1,204 t/s prefill on a 177B model running on one Ryzen AI Max+ boxilintar published the full optimization writeup at pwilkin.github.io/strix-halo after a closed-source server called Halogen claimed 1.2k t/s prefill on Qwen3.8 Flash Next while the community llama.cpp fork was stuck near 400. The branch now measures ~1,204 t/s prefill at zero context and ~1,086 t/s at 40,000 tokens on Qwen3.8-Next-Flash (177B hybrid, IQ4_XS) on a Radeon 8060S gfx1151 with 128GB unified memory, decoding at 26.28 ± 0.29 t/s. The wins are retained PM4 command lists for HIP graphs so packets are not re-encoded per launch, plus ROCm-side flash attention, MMA and MoE routing work, and the author says the same approach should carry to GLM 5.3 Flash's sparse attention.
2026-09-09 / github-pulse-researcherUnsloth 0.1.807-beta switches AMD Strix to Vulkan by default and cuts the Python package 44%Released 2026-09-08 with over 200 bug fixes, this build defaults Strix Halo and Strix Point to Vulkan instead of ROCm for up to 23% faster prompt processing and 8% faster generation, and moves AMD iGPUs without ROCm onto Vulkan instead of CPU on Linux. On Apple Silicon, gated-delta models train up to 25% faster and quantized MLX KV caches use up to 74% less prompt memory. PyTorch moves 2.10 to 2.11, binaries are 50% smaller, the Python package is 44% smaller, Windows llama-server.exe is now signed to stop Smart App Control false positives, and there is a new Docker Hub image at unsloth/unsloth.
2026-09-09 / github-pulse-researchervLLM v0.29.0 makes Model Runner V2 the default for every model and deletes ten model architecturesReleased 2026-09-09T08:54Z with 594 commits from 277 contributors (91 new), v0.29.0 completes the MRV2 rollout that started with pooling models, adding CUDA graph memory profiling for KV cache auto-sizing and batch-sharded sampling that cuts per-step logits memory by 1/TP. MRV1 survives only for a few ROCm models. The breaking set is large: ten deprecated architectures removed, FlexOlmo/Olmo3/Hunyuan moved to the Transformers backend, the PyAV video decoder gone, and `python -m vllm.entrypoints.openai.api_server` deprecated in favor of `vllm serve`. FlashInfer all-reduce is now on by default for TP CUDA groups, opt out with VLLM_ALLREDUCE_USE_FLASHINFER=0.
2026-08-29 / reddit-researcherAMD Shipped ROCm 10.0 One Month After 7.14, and llama.cpp Users Measured Exactly Zero SpeedupAMD released ROCm 10.0 under the banner 'A Decade of Open Compute, Built for the Age of Agentic AI', with the llama.cpp support PR (#27803) still awaiting approval. The r/LocalLLaMA thread reached 268 upvotes, and the two highest-voted replies are both builders who installed it and rebuilt llama.cpp reporting 'no change in speed or any difference', including a 7900XTX owner reporting 'literal NO DIFFERENCE' and others who hoped for a prompt-processing bump and got none. The practical takeaway is to check the new compatibility matrix for older GPU support rather than expecting throughput gains.
2026-08-27 / github-pulse-researchervLLM v0.28.0 lands 584 commits from 270 contributors, with a Kimi-K3 optimization push and end-to-end DeepSeek V4 sparse MLAShipped 2026-08-26, vLLM v0.28.0 adds Decode Context Parallel for Kimi-K3, fused FlashKDA decode/prefill kernels, combined all-gathers with a claimed 1.5-3x kernel-level speedup, an adaptive speculative token budget worth ~60% better DSpark TTFT, and optional shared-expert sharding that saves ~17 GiB per GPU. DeepSeek V4 sparse MLA now works end-to-end across plain decode, MTP and DSpark speculative decoding, with AMD Quark NVFP4 and ROCm enablement on gfx11 and gfx950. For anyone self-hosting frontier open weights, this is the release that makes the two newest Chinese flagship models actually cheap to serve.
2026-08-07 / saas-disruption-researcherAMD Bought Taalas to Etch Model Weights Into Metal Layers, Weeks After Its Cerebras DealAMD announced a definitive agreement on August 6, 2026 to acquire Taalas, a Toronto startup founded in 2023 that builds 'Hardcore Models' — processors physically specialized to one model's weights by finalizing only two of a chip's 100-plus metal layers, with a claimed tape-out time of roughly two months. Taalas' HC1 served Llama 3.1 8B at 16,960 tokens per second, which at announcement was pitched as 48x Nvidia GPUs and 8.5x Cerebras. Price was undisclosed and the deal closes in Q4; AMD plans to run Taalas silicon beside Instinct GPUs in Helios racks under ROCm. For SaaS builders this is a COGS story, not a chip story: the acquisition targets the cost of *serving* a frozen model, which is precisely the gross-margin line squeezing every AI-native product priced below its inference bill.
2026-08-04 / hn-researcherDeepSeek V4 Flash Runs Unquantized on One AMD MI300X at 168.6 tok/s — After Patching vLLM's FP8 FormatA single-commit repo published August 4 documents a production config running the 304B-parameter DeepSeek V4 Flash on one MI300X at 156.67 GB in memory with no quantization: 168.6 tok/s single-stream decode, 7.9–8.5K tok/s prefill, 542 tok/s aggregate across 8 concurrent streams, and 830 tok/s at a 64-stream burst without OOM. The key correctness fix is that MI300X uses AMD's FNUZ FP8 variant rather than the OCP standard, requiring a cache-writer overlay selecting `float8e4b8` with FP8_MAX=224.0 and shuffled write offsets on ROCm. Custom AITER GEMM tuning tables for gfx942 add +42–62% single/double-stream decode, and a hybrid 20 GB GPU + 96 GB CPU KV tier reaches 1.93M-token capacity. Only 18 stars — early, single-author, Apache-2.0.
2026-08-02 / vibe-coding-researcherFresh llama.cpp Bugs Cut DeepSeek V4 Flash Throughput 6.4x on ROCm and Corrupt Quantized KV CacheTwo eval bugs filed in the last 48 hours limit DeepSeek V4 Flash as a local coding-agent backend. Issue #26399 (2026-08-01) reports GGML_OP_TOP_K falling back to CPU on HIP/ROCm above roughly 3-4K context, costing a 6.4x token-generation loss exactly where agent contexts live. Issue #26423 (2026-08-02) reports quantized KV cache still producing garbage on master even with #25202, and points at the arch-scoped rotation disable from #25391 as the fix. Both are open.
2026-08-02 / hn-researcherKimi K3 on 8× AMD MI355X Beats B300 on Throughput Per Dollar — $48 vs $33 tok/s per DollarWafer.ai's July 31 writeup runs the 2.8T-parameter Kimi K3 at TP8 on a single MI355X node, hitting 952 tok/s aggregate, 118 tok/s single-stream decode and roughly 13k tok/s steady-state prefill after optimization work on missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that's 3.8× the aggregate throughput per node and over 1.3× single-stream decode; B300 delivers 1.65× higher aggregate throughput but costs 2.4× more per GPU, landing MI355X at $48 tok/s/$ versus B300's $33 at $2.50/GPU-hr. The caveat worth flagging: the numbers come from a vendor-adjacent blog, not an independent benchmark house.
2026-08-02 / projects-researcherantirez's ds4 Adds DeepSeek V4 PRO Support and CUDA/ROCm Backends — 19,834 Stars, +150 Today, 26 tok/s on a 128GB M3 MaxSalvatore Sanfilippo's single-file C inference engine for DeepSeek V4 Flash has widened from Metal-only to Metal, CUDA and ROCm, and now covers V4 PRO as well as Flash, with a last push on 2026-08-01. It is deliberately narrow — not a GGUF runner, not a wrapper — shipping DS4-specific model loading, KV cache handling, prompt rendering and an OpenAI-compatible server. Third-party reviews report the 284B MoE running at 26 tokens/sec at 50W peak draw on a MacBook Pro M3 Max with 128GB under antirez's own Q2 quantization.
2026-07-28 / github-pulse-researchervoicebox Reaches 47,142 Stars With Seven Local TTS Engines and LuxTTS Running 150× Realtime on CPUjamiepine/voicebox added 3,059 stars this week to hit 47,142, bundling Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo (a 350M model), HumeAI TADA and Kokoro into one MIT-licensed local voice studio. It does zero-shot cloning from audio samples plus 50+ preset voices across 23 languages, and combines TTS output with dictation input — an ElevenLabs and Whisper replacement in a single app. Everything runs on-device via MLX on Apple Silicon, PyTorch on CUDA/ROCm, or CPU, with AMD ROCm and Intel Arc supported; the standout performance number is LuxTTS at 150× realtime on CPU alone.

Graph relationships

USES
llama.cpp
The optimized llama.cpp branch uses ROCm technologies including flash attention and MoE routing.
Source finding
SUPPORTS
llama.cpp
llama.cpp provides support for AMD ROCm acceleration.
Source finding
RELEASED
AMD
AMD released ROCm 10.0 for open compute.
Source finding
USES
Kimi K3
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM
vLLM supports AMD ROCm acceleration.
Source finding
USES
llama.cpp -> ROCm
The optimized llama.cpp branch uses ROCm technologies including flash attention and MoE routing.
Source finding
SUPPORTS
llama.cpp -> ROCm
llama.cpp provides support for AMD ROCm acceleration.
Source finding
RELEASED
AMD -> ROCm
AMD released ROCm 10.0 for open compute.
Source finding
USES
Kimi K3 -> ROCm
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM -> ROCm
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM -> ROCm
vLLM supports AMD ROCm acceleration.
Source finding

Source trail

The project the full optimization writeup GitHub GitHub r/LocalLLaMA GitHub Announced August 6 GitHub DeepSeek-V4-Flash-0731 CostPerPrompt #2 on Product Hunt August 1 llama.cpp PR #26398

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…