Entity trail

Qwen3.8-Flash-Next

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / reddit-researcherQwen3.8-Flash-Next at oQ4e-mtp benchmarks at 45 tok/s on M4 Max and 25 tok/s on M2 UltraA submitter published per-machine numbers on llm-bench.io showing Qwen3.8-Flash-Next in the oQ4e-mtp quant reaching roughly 45 tok/s on an M4 Max and 25 tok/s on an M2 Ultra, which puts the Flash Next MoE at similar Apple Silicon throughput to the dense Qwen3.8-27B. A commenter flagged that the page shows a 46.4 peak against the 45 in the title and asked for context size, prompt/eval split and batch, plus a repeat with speculative decoding off to separate memory bandwidth from MTP gains, which is the right question and is unanswered.
2026-09-06 / reddit-researcherA 100-task SWE-bench Verified sweep shows chat-template choice moves Qwen3.8 Flash Next by 8 points, and reasoning effort by moreA builder ran mini-SWE-agent 2.4.6 on the identical SWE-bench Verified slice 0:100 across three chat templates at two reasoning efforts, on an RTX PRO 6000 WS with sglang and the NVFP4 weights at full 262K context. Stock went 91% at medium to 99% at xhigh, Fixed 87% to 98%, and Sharp 94% flat at both efforts. The tradeoff is time and tokens: stock xhigh cost +143.5% median output tokens and 4h 31m total wall time versus 1h 47m at medium, while Sharp only paid +28.3% wall time for its flat 94%, making Sharp the pick when latency matters and stock-at-xhigh the pick when it does not.
2026-09-05 / github-pulse-researcherllama.cpp 0.4.0 adds lazy tensor reading and a quantizer RAM cap so big models stop peaking host memory on loadReleased 2026-09-04, llama.cpp 0.4.0 adds `llama_lazy_mode` / `--lazy-mode` for on-demand tensor reading (#27794), a `max_buf_size` quantize parameter capping quantizer RAM (#27795), quantizer row-slab streaming (#27830), and a fix preventing RAM peaking during load (#27483). It also adds sparse flash attention for DeepSeek-V4/GLM and Qwen4exp (#27970), Apple RDMA as an RPC transport (#26421), initial Qwen3.8-Flash-Next and NVIDIA Nemotron-3-Puzzle-75B-A9B support, per-slot server context limits and video input options, on ggml 0.23.0.
2026-09-04 / reddit-researcherQwen3.8-Flash-Next multi-token prediction merged into ik_llama.cpp, roughly doubling decode on a 5090The merge supports both an integrated MTP head and a separate -md draft file, with the poster reporting 45 to 90 tok/s on a 5090 with 128GB system RAM and working configurations down to a 12GB 4070. A separate Part 2 post on the 2x3090 plus DDR4 build reports 37-41 t/s decode using UD-Q4_K_XL plus expert cache plus MTP, up from the 25-29 t/s that same builder reported two days ago. Both are single-builder measurements on hand-built branches, not vendor numbers.
2026-09-03 / reddit-researcherAn expert-cache PR lifts Qwen3.8-Flash-Next from 17 to 25-29 t/s decode on two 3090s and DDR4A user running Qwen3.8-Flash-Next at unsloth UD-Q6_K_XL on 2x RTX 3090 over PCIe 3.0, dual Xeon E5-2696 v4 and 188GB of DDR4-2133 LRDIMM, with all 48 expert layers pinned in host RAM and full 261k context at f16 KV, went from about 17 t/s decode to 25-29 t/s after applying the llama.cpp expert cache PR. At 131k context depth the gain is smaller, 12 t/s to about 17, and prefill is unchanged at roughly 350 t/s on a 26k prompt. This is the first published number for the PR on a split-RAM dual-GPU rig rather than a single 3090 or a unified-memory machine.
2026-09-02 / github-pulse-researcherUnsloth 0.1.805-beta doubles Qwen3.8-Flash-Next and GLM-5.3-Flash inference with MTP on by defaultReleased 2026-09-02, Unsloth v0.1.805-beta enables multi-token prediction by default for Qwen3.8-Flash-Next and GLM-5.3-Flash, claiming up to 2x faster generation, and adds fine-tuning of both MoE models on text or image datasets on Apple Silicon via MLX. Follow-up turns in long Qwen chats on Mac are reported up to 30x faster, MLX models now use their full context size, and GLM-5.3 MLX fine-tunes export to GGUF. Local models can also now use Codex's `apply_patch` tool for code edits.
2026-09-02 / projects-researcherslotstream runs a 104GB model on a 48GB Mac at 12 tok/s by streaming MoE experts off SSD with preadcarloslfu/slotstream hit 220 points on Hacker News overnight and holds 210 stars, MIT, Swift and MLX, created August 28. It runs Qwen3.8-Flash-Next, a 125B MoE weighing 103.8GB across 24 files at 4-bit, by loading only the 3.8GB dense trunk into RAM (about 2 second startup) and reading routed experts with pread into a fixed pool of cache slots shared across all 48 layers, so hot layers borrow slots from cold ones. It avoids mmap because MLX cannot materialize part of a memory-mapped tensor, which would force the full ~100GB. Published throughput scales 3 tok/s at 8GB, 4 at 16GB, 8 at 24GB, 12 at 48GB and up, needing ~110GB free disk.
2026-09-01 / arxiv-researcherQwen3.8-Flash-Next Matches a 397B Predecessor on 8 of 14 Benchmarks Using a Ninth of the Training FLOPsThe architecture report describes a 125B sparse mixture-of-experts model activating 6B parameters per token, plus 51B of n-gram embedding tables held off the accelerator in host memory with prefetching. Against the prior 397B-A17B model it leads on 8 of 14 pre-training benchmarks and trails by at most 2.6 points elsewhere, using a third of the activated parameters, a third of the training tokens and roughly a ninth of the training FLOPs. The design mixes Gated DeltaNet with one full-attention layer per four, adds Qwen Sparse Attention for continued pretraining and a four-branch Gated Residual stream, and reports that pairing it with Muon shifts optimal learning rate and batch size upward and removes the need for batch-size warmup.
2026-09-01 / github-pulse-researcherllama.cpp replaces a transpose-plus-sum_rows indexer reduction with strided views, worth 9% prefill on an RTX PRO 6000Build b10730 (#28023), published 2026-09-01, rewrites the qwen4exp indexer head reduction that went through a transpose and a sum_rows over ne[1], leaving sum_rows with ne0 = 4 — one block per row for a four-element reduction — while the transpose copied the whole block-by-token surface twice on the way in. Because the heads are adjacent on ne[1], each is now a strided view and the sum is a short chain of adds. On an RTX PRO 6000 with Qwen3.8-Flash-Next UD-Q4_K_XL, fa on and 55k context, prompt processing goes from 2,170 to 2,366 t/s with generation unaffected, and the gain grows with context and ubatch size.
2026-09-01 / github-pulse-researcherllama.cpp adds recurrent state rollback for Qwen3.8-Flash-Next, taking MTP speculative decoding from 123 to 183 tok/sBuild b10731, published 2026-09-01, implements recurrent state rollback for qwen4exp (#28123), which MTP speculative decoding needs so the target state can move back by the number of rejected draft tokens. Without it the context was classified SEQ_RM_TYPE_FULL and the server serialized the whole recurrent state to host memory every round, costing more than the drafting saved. Measured on Qwen3.8-Flash-Next UD-Q4_K_XL with n-max 3 and a single slot, decoding reaches 183 tok/s on code and 144 on prose, against 123 and 83 on the same branch before the change and 108 tok/s with no draft at all.
2026-08-31 / reddit-researcherQwen3.8-Flash-Next runs at 3.5 tok/s on a 12GB mid-range Android phoneA video post to r/LocalLLaMA shows the 80GB Qwen3.8-Flash-Next generating at 3.5 tokens per second on a 12GB mid-range Android handset, using aggressive quantization on the dense part plus unnamed optimizations, on a device the author puts at $400 to $500. The CPU hits 80C during the run, which the top commenter flags immediately, so this is a demonstration of what fits rather than something you would leave running. It is the same sparse-MoE-plus-offload trick that makes Flash-Next viable on constrained desktops, pushed to a phone.
2026-08-31 / reddit-researcherA head-to-head swap of Qwen3.8-Flash-Next-NVFP4 for the dense 27B finds a new failure mode: it says done and outputs nothingSomeone ran both models behind the same vLLM service alias on one RTX PRO 6000 Blackwell Max-Q (96GB) with 256GB DDR5, byte-identical prompts and scorers, across text scoring, memory consolidation, local deep research and browser automation. Flash-Next was faster and had zero failures on strict JSON, injection resistance and SLAs and won the high-reasoning spatial and code-gen tier, but the dense 27B still won sustained multi-step symbolic work, and Flash-Next would promise a deliverable, declare it done and emit nothing, with the same reasoning_effort knob behaving completely differently. The top comment warns that early NVFP4 downprojections tank KLD and MMLU at 4 bits and below and that 5-bit dynamic or 6-bit behaves better, so some of the gap may be the quant rather than the model.

Graph relationships

BENCHMARKED AGAINST
M2 Ultra
Qwen3.8-Flash-Next reaches 25 tok/s on M2 Ultra
Source finding
BENCHMARKED AGAINST
M4 Max
Qwen3.8-Flash-Next reaches 45 tok/s on M4 Max
Source finding
USES
slotstream
slotstream runs Qwen3.8-Flash-Next model on macOS by streaming experts off SSD.
Source finding
SUPPORTS
Unsloth
Unsloth enables multi-token prediction by default for Qwen3.8-Flash-Next
Source finding
USES
SGLang
Qwen3.8-Flash-Next runs in SGLang with SSD-streamed n-gram table optimization.
Source finding
COMPETES WITH
GLM-5.3-Flash
Qwen3.8-Flash-Next leads Hugging Face trending with 3x score of GLM-5.3-Flash
Source finding
USES
n-gram embeddings
Per-layer 97.7 GiB n-gram embeddings table can be disk-resident
Source finding
SUPPORTS
llama.cpp
PR 27742 merged Qwen3.8-Flash-Next with memory-mapped n-gram table support
Source finding
SUPPORTS
Unsloth
Unsloth beta added local support for Qwen3.8-Flash-Next
Source finding
RELEASED
Alibaba
Alibaba is shipping Qwen3.8-Flash-Next as a technology preview of Qwen4 architecture.
Source finding
SUPPORTS
Unsloth
Unsloth released day-0 quantizations for Qwen3.8-Flash-Next.
Source finding
RELEASED
Hugging Face
Qwen3.8-Flash-Next was released on Hugging Face with estimated drop on 2026-08-26.
Source finding

Source trail

llm-bench.io (via r/LocalLLaMA)r/LocalLLaMA (58 upvotes)GitHub r/LocalLLaMA (162 upvotes)r/LocalLLaMA GitHub Hacker News arXiv 2608.30320 GitHub GitHub r/LocalLLaMA r/LocalLLaMA

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…