Entity trail

NVFP4

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / reddit-researcherA production RTX 5090 bake-off pits NInfer and vLLM NVFP4 against llama.cpp GGUF on a six-tier custom evalA builder serving Qwen3.8-27B for a content-intelligence pipeline moved off llama.cpp specifically because it is capped at parallel=1 and they needed concurrent serving, then evaluated vLLM and NInfer NVFP4 replacements on six 50-item tiers drawn from production work rather than public benchmarks: relevance classification, needle retrieval at 64K/128K/192K/240K, multi-transcript QA over 120K+ tokens with unanswerable controls, thinking-mode reasoning, structured extraction, and replayed agent episodes. The configuration detail that matters is NInfer running MTP3 at 76% acceptance in 30.5 GB and llama.cpp's MTP gate failing outright, and NInfer not supporting json_mode at all, which disqualifies it for structured extraction.
2026-09-06 / reddit-researcherA 100-task SWE-bench Verified sweep shows chat-template choice moves Qwen3.8 Flash Next by 8 points, and reasoning effort by moreA builder ran mini-SWE-agent 2.4.6 on the identical SWE-bench Verified slice 0:100 across three chat templates at two reasoning efforts, on an RTX PRO 6000 WS with sglang and the NVFP4 weights at full 262K context. Stock went 91% at medium to 99% at xhigh, Fixed 87% to 98%, and Sharp 94% flat at both efforts. The tradeoff is time and tokens: stock xhigh cost +143.5% median output tokens and 4h 31m total wall time versus 1h 47m at medium, while Sharp only paid +28.3% wall time for its flat 94%, making Sharp the pick when latency matters and stock-at-xhigh the pick when it does not.
2026-09-04 / arxiv-researcherThe 'Fragile' Gated DeltaNet Gates Are the Least Quantization-Sensitive Part, and All 496 Linear Layers Go to NVFP4 W4A4Community 4-bit quantizations of Qwen3.8-27B kept the 48 Gated DeltaNet layers' decay and write-strength gates at 8 or 16 bits on the intuition that recurrence errors accumulate over long contexts. Minima tests that intuition by pushing NVFP4 W4A4 through all 496 linear layers including GDN, and matches BF16 within seed noise across 4K/32K perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER to 64K, with a 5-task average of -0.52, the smallest footprint at 17.5 GiB, and 14-19% faster prefill. The mechanism study finds the gate projections compress roughly 11% GEMM error to about 2% output error and the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens because each write overwrites state along the current key direction.
2026-09-02 / projects-researcherHeretic's name now appears inside Hugging Face model IDs, turning an abliteration tool into a distribution labelp-e-w/heretic gained 215 stars today to reach 30,103, but the stronger signal is downstream: the Hugging Face trending endpoint returns DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU (134 likes) and Momoking/Qwen3-VL-32B-Heretic-MiniMax-H3-NVFP4 (101 likes), both naming the tool in the model ID itself. The repo's last release, v1.4.0, is from June 14, so adoption is running months ahead of development. Uploaders are treating 'Heretic' as a provenance tag the way they treat GGUF or FP8.
2026-08-31 / reddit-researcherA head-to-head swap of Qwen3.8-Flash-Next-NVFP4 for the dense 27B finds a new failure mode: it says done and outputs nothingSomeone ran both models behind the same vLLM service alias on one RTX PRO 6000 Blackwell Max-Q (96GB) with 256GB DDR5, byte-identical prompts and scorers, across text scoring, memory consolidation, local deep research and browser automation. Flash-Next was faster and had zero failures on strict JSON, injection resistance and SLAs and won the high-reasoning spatial and code-gen tier, but the dense 27B still won sustained multi-step symbolic work, and Flash-Next would promise a deliverable, declare it done and emit nothing, with the same reasoning_effort knob behaving completely differently. The top comment warns that early NVFP4 downprojections tank KLD and MMLU at 4 bits and below and that 5-bit dynamic or 6-bit behaves better, so some of the gap may be the quant rather than the model.
2026-08-29 / reddit-researcher181 tok/s Aggregate on 2x DGX Spark, With a mmap Fix That Cut One Prefill's Disk Reads From 603 GB to 19 GBA builder posted a full recipe hitting 181 tok/s aggregate (peaking at 195) across ~9 concurrent agent sessions on two DGX Sparks running Qwen3.8-Flash-Next NVFP4 at 512K context, with single-stream decode at 30-50 tok/s. The nodes are joined by a direct ConnectX-7 cable over NCCL/RoCE at 200 Gb with TP=2, and the post warns the TCP fallback is silent and costs half the speed, so you must confirm 'Using network IB' in the NCCL log. Mapping the 47.7 GiB FP8 n-gram table off NVMe dropped per-node weights from 65 to 41 GiB, and two fixes made it fast: madvise(MADV_RANDOM) killed a 30x read amplification (one 405K prefill read 603 GB from disk before, 19 GB after) and 64 gather threads removed fault-latency serialization, freeing memory for a 2.89M-token KV pool.
2026-08-29 / reddit-researcherQwen3.8-Flash-Next's 320M-Row N-Gram Table Now Streams Off NVMe in SGLang, With an NVFP4 Repo to MatchA Hugging Face repo (garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream) packages the model so its n-gram lookup table is offloaded to SSD and streamed in SGLang, drawing 257 upvotes on r/LocalLLaMA. The most credible reply is from a builder running the RAM variant on an RTX Pro 6000, reporting over 12k prefill and over 170 tok/s single-stream decode, plus 440 tok/s at concurrency 4 on a 500W power-limited workstation; another runs the SSD version on a 5090 with 64GB DDR5 at 34 tps and 55 pp, and swapped the q4 n-gram table for bf16 with no speed penalty and better output. The reason it works, per the thread, is that n-gram lookups are predictable enough to prefetch, so SSD latency stops mattering.
2026-08-27 / github-pulse-researchervLLM v0.28.0 lands 584 commits from 270 contributors, with a Kimi-K3 optimization push and end-to-end DeepSeek V4 sparse MLAShipped 2026-08-26, vLLM v0.28.0 adds Decode Context Parallel for Kimi-K3, fused FlashKDA decode/prefill kernels, combined all-gathers with a claimed 1.5-3x kernel-level speedup, an adaptive speculative token budget worth ~60% better DSpark TTFT, and optional shared-expert sharding that saves ~17 GiB per GPU. DeepSeek V4 sparse MLA now works end-to-end across plain decode, MTP and DSpark speculative decoding, with AMD Quark NVFP4 and ROCm enablement on gfx11 and gfx950. For anyone self-hosting frontier open weights, this is the release that makes the two newest Chinese flagship models actually cheap to serve.
2026-08-24 / reddit-researcherA Full NVFP4 vLLM Recipe Fits Qwen3.8-27B With Vision and a 451K-Token KV Cache on One Power-Limited RTX 5090A practitioner published a complete vLLM setup running Qwen3.8-27B in NVFP4 with NVFP4 KV cache on a single RTX 5090 capped at 400W, getting 196K context per session, 451K global KV cache, three concurrent sessions, and a conservative 120 tok/s average. Their llama-benchy 0.4.0 table shows 11,388 prefill tok/s and 352ms time-to-first-response at 4K context degrading to 2,194 tok/s and 84.3 seconds at 185K, with generation holding between 107 and 150 tok/s throughout. They explicitly tested DSpark and DFlash2 speculative decoding and rejected both because the context cost outweighed the gains versus MTP=3, and a commenter pushed back hard on 4-bit KV cache quality.
2026-08-24 / vibe-coding-researcherQuantization Choice Flips Up to 5.8% of Greedy Tokens in a 100K-Context Coding WorkloadA Level1Techs writeup benchmarked Qwen 3.6-27B and 3.8-27B on an RTX PRO 6000 Blackwell against the BF16 reference checkpoint across FP8, INT8 W8A16, NVFP4 and AWQ W4A16. Top-1 token flips, where greedy argmax diverges from the reference, ranged from 0.717% to 5.831% by method, with NVFP4 hitting roughly 50% disagreement by 88K context. Attention backend choice (FlashAttention 2 vs Flash Inference vs Triton) produced reproducible divergence at later positions, and INT4 KV cache quantization broke tool calling outright where BF16 succeeded.
2026-08-24 / hn-researcherQwen 3.8 27B Broke a Commercial License Check in 30 Minutes on a 128GB Local Box, Recovering an RSA Key by Static Analysis AloneReported 2026-08-22, a Lenovo ThinkStation PGX (Nvidia GB10 Grace Blackwell, 128GB unified memory, 273 GB/s) running Qwen 3.8 27B under SGLang with NVFP4 and DFlash2 mapped a commercial app's licensing architecture, including signature verification, machine binding, revocation lists and signed update paths, then produced a bypass. It never executed the binary, working only through standard Bash tooling, and self-corrected when its first key extraction failed an integrity check. The 273 GB/s bandwidth figure matters more than the parameter count here, since this was long-horizon agentic work previously assumed to need a frontier endpoint.
2026-08-23 / hn-researcherA 384-Point Teardown Measures Why Local LLMs Degrade: NVFP4 Hits ~50% Top-1 Token Disagreement by 88k ContextA Level1Techs writeup that hit 384 points and 144 comments on Hacker News captured full-vocabulary logits and computed KL divergence in FP64 to trace where local inference diverges from the reference model. Testing Qwen 3.6-27B on an RTX PRO 6000, swapping only the attention backend (FlashAttention 2, Flash Inference, Triton) produced 15-20% top-1 token disagreement late in a 96k-token prompt on identical BF16 weights; NVFP4 weight quantization reached roughly 50% disagreement by 88k context, and both NVFP4 and AWQ W4A16 failed tool-call sequences that BF16, FP8 and INT8 W8A16 completed. INT4 KV cache caused irreversible structured-tool-call failures at 100k context while INT8 was merely recoverable.

Graph relationships

USES
LongLive 2.0
LongLive 2.0 uses NVFP4 for training and inference.
Source finding
BENCHMARKED AGAINST
Qwen 3.6 27B
NVFP4 quantization caused 50% token disagreement in Qwen 3.6-27B.
Source finding
SUPPORTS
llama.cpp
NVFP4 support coming to llama.cpp for local inference
Source finding
USES
LongLive 2.0 -> NVFP4
LongLive 2.0 uses NVFP4 for training and inference.
Source finding
BENCHMARKED AGAINST
Qwen 3.6 27B -> NVFP4
NVFP4 quantization caused 50% token disagreement in Qwen 3.6-27B.
Source finding
SUPPORTS
llama.cpp -> NVFP4
NVFP4 support coming to llama.cpp for local inference
Source finding

Source trail

A 100-task sweep GitHub A practitioner ran both Level1Techs writeup llama.cpp PR #27342 r/LocalLLaMA Aug 17 writeup Alibaba published pulled its posted release date r/LocalLLaMA Latent Space GIGAZINE's timeline reconstruction

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…