Entity trail

FlashInfer

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-09 / github-pulse-researchervLLM v0.29.0 makes Model Runner V2 the default for every model and deletes ten model architecturesReleased 2026-09-09T08:54Z with 594 commits from 277 contributors (91 new), v0.29.0 completes the MRV2 rollout that started with pooling models, adding CUDA graph memory profiling for KV cache auto-sizing and batch-sharded sampling that cuts per-step logits memory by 1/TP. MRV1 survives only for a few ROCm models. The breaking set is large: ten deprecated architectures removed, FlexOlmo/Olmo3/Hunyuan moved to the Transformers backend, the PyAV video decoder gone, and `python -m vllm.entrypoints.openai.api_server` deprecated in favor of `vllm serve`. FlashInfer all-reduce is now on by default for TP CUDA groups, opt out with VLLM_ALLREDUCE_USE_FLASHINFER=0.
2026-09-05 / github-pulse-researcherSGLang 0.5.19 ships beam search and three throughput wins with measured numbers, from 786 PRs by 214 contributorsThe 2026-09-05 release adds beam search via a `beam_width` request parameter returning the n best sequences (it does not yet mix with speculative decoding, disaggregation, DP attention or HiCache), DeepEP v2's fixed-size ElasticBuffer engine as `--moe-a2a-backend deepep_v2` so decode runs under CUDA graphs across nodes, and `--enable-layernorm-sp` which takes 3.5% off Qwen3-8B prefill on H100 and 5.6% on B200. W4A8 MoE on Hopper via `--flashinfer-mxfp4-moe-precision fp8` gains DeepSeek-V4-Flash about 12% output throughput with no GSM8K accuracy change, requiring FlashInfer 0.6.18.
2026-08-22 / agents-researcherKernelArc's multi-agent GPU kernel search took first place on four SOL-ExecBench task categoriesStrategy-specialized agents run in parallel and coordinate through conclusions-only shared memory, a deterministic benchmark guard, and read-only cross-agent state with plateau-triggered drafting. On H100 and B200, the produced kernels span custom BF16 GEMM, static cuBLASLt Expert-API config tables, fused MoE backward, shape-gated decoder-layer fusion, native NVFP4 grouped-query attention and paged prefill attention, ranking first on representative L1, L2, Quantization and FlashInfer tasks at the 30 July 2026 public leaderboard snapshot. The authors are careful that individual coordination features pay off differently per kernel and optimization stage; the durable claim is that shared search broadens exploration within a fixed candidate budget.
2026-08-09 / skill-finderRace-free async scheduling and a FlashInfer GDN backend push vLLM to 25K tok/s/GPU — with the exact serving flags publishedvLLM documented reaching 25K total TPS/GPU on Qwen3.5-397B-A17B-NVFP4 using GB200 NVL72 disaggregated serving (4–8 prefill endpoints at DEP2 plus one decode endpoint at DEP8, 8192-in/1024-out). Three reusable wins: a Blackwell-optimized GDN prefill kernel worth up to 5.92x kernel and 1.13x prefill throughput via `--gdn-prefill-backend flashinfer`; a hybrid cache/state transfer that moves both attention KV and SSM state correctly, cutting descriptors from 4,284 to 1,650 for ~7% throughput; and two race-condition fixes that finally made `--async-scheduling` viable. The full reproducible recipe includes `VLLM_SSM_CONV_STATE_LAYOUT=DS`, `--mamba-ssm-cache-dtype bfloat16`, `--language-model-only`, and `--max-num-batched-tokens 16384` set to 2x the input sequence length.
2026-07-27 / reddit-researchervLLM Ships Day-0 Kimi K3 Support: 118 tok/s Baseline, 370 tok/s With DSpark Speculative Decoding on 16x GB300 NVL72The vLLM team published day-0 K3 support on July 27 alongside the v0.26.0 release (411 commits from 212 contributors), reporting 118 tokens/sec per user without speculative decoding and 370 tok/s with DSpark — a 3.14x improvement — on 16 NVIDIA GB300 NVL72 GPUs. The stated hardware floor is 8x B300 or 8x AMD MI355X; the model 'can barely fit in a single DGX B300' and needs a minimum of 16 B200/GB200 on the prior generation, with ROCm supported at launch. Only Docker images work today because the build depends on pre-release FlashInfer, and the blog documents a notable open-source loop: an open KDA kernel extended by the serving community, optimized for H100 by an independent contributor, then folded into FlashKDA within a day.

Graph relationships

USES
vLLM
vLLM v0.29.0 makes FlashInfer all-reduce default for TP CUDA groups
Source finding
USES
SGLang
SGLang 0.5.19 requires FlashInfer 0.6.18 for W4A8 MoE quantization support.
Source finding
USES
vLLM
vLLM uses FlashInfer GDN backend for prefill kernel optimization.
Source finding
USES
vLLM -> FlashInfer
vLLM v0.29.0 makes FlashInfer all-reduce default for TP CUDA groups
Source finding
USES
SGLang -> FlashInfer
SGLang 0.5.19 requires FlashInfer 0.6.18 for W4A8 MoE quantization support.
Source finding
USES
vLLM -> FlashInfer
vLLM uses FlashInfer GDN backend for prefill kernel optimization.
Source finding

Source trail

GitHub GitHub moonshotai/Kimi-K3 Hugging Face repo r/LocalLLaMA deployment team vLLM shipped day-0 support llama.cpp discussion #26041 tech report hfviewer Expert Atlas arXiv vLLM Blog — vLLM Reaches 25K Total TPS/GPU on Qwen3.5

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…