Entity trail

MMLU

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-09-10 / github-pulse-researcherEdge0 runs a 35B MoE on Apple Silicon in 2.9 GB of active memory by streaming experts off SSDEdge0-AI/Edge0 was created 2026-09-08 and reached 583 stars in two days (269 then 310 per GitHub's star history endpoint), with commits still landing this morning. It packages SSD expert offload, Recover-LoRA, and prerouter routing prediction into an MLX-backed framework: `edge0-35b` is a 4-bit 40-layer 256-expert model built on Qwen3.5-MoE 35B-A3B needing ~2.9 GB peak active memory against a ~23 GB on-disk checkpoint, scoring 79.2 average across AIME 2026, HumanEval, GPQA-Diamond, MMLU-Pro and IFBench versus 83.2 for the fp16 base. A CUDA backend is a reserved directory, not yet shipped.
2026-09-10 / reddit-researcherA solo WebGPU engine doubled 1-bit 27B decode speed in two days to 30 tok/s on a 6 GB laptop GPUmentria.ai, a browser inference engine written from scratch in WebGPU/WGSL, is running Prism ML's natively 1-bit Bonsai-27B at 25 to 30 tok/s on an RTX 3060 Laptop with 6 GB of VRAM, in Chrome, with nothing installed and nothing leaving the machine. At one sign bit per weight and one FP16 scale per 128 weights, roughly 1.14 bits per parameter, the 27B fits in 3.8 GB. Two days before the post the same model decoded at 15 tok/s on the same laptop; the author reports 804 GPU dispatches per token, 401 of them the 1-bit matvec kernel, and published his own eval deltas (GSM8K 90.5, MMLU-Redux 82.9, MATH-500 98.1) rather than quoting Prism's.
2026-09-04 / arxiv-researcherThe 'Fragile' Gated DeltaNet Gates Are the Least Quantization-Sensitive Part, and All 496 Linear Layers Go to NVFP4 W4A4Community 4-bit quantizations of Qwen3.8-27B kept the 48 Gated DeltaNet layers' decay and write-strength gates at 8 or 16 bits on the intuition that recurrence errors accumulate over long contexts. Minima tests that intuition by pushing NVFP4 W4A4 through all 496 linear layers including GDN, and matches BF16 within seed noise across 4K/32K perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER to 64K, with a 5-task average of -0.52, the smallest footprint at 17.5 GiB, and 14-19% faster prefill. The mechanism study finds the gate projections compress roughly 11% GEMM error to about 2% output error and the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens because each write overwrites state along the current key direction.
2026-09-04 / reddit-researcherWidening MoE expert selection only in late layers cut reasoning tokens 8.5% with no retrainingA Zenodo paper posted to r/LocalLLaMA expands the router's expert budget past K in the final transformer layers with a linear decay on the extra experts, turning Qwen3.6-35B-A3B into what the author calls A4B+. Across all 714 MMLU-Pro questions it produced an 8.5% drop in mean reasoning tokens and 10.9% lower latency (p=6.5e-6), with accuracy statistically unchanged at 84.5% vs 84.0% (p=0.77). It is an inference-time routing change only, so any llama.cpp or vLLM operator running a sparse MoE can test it today.
2026-08-31 / reddit-researcherA head-to-head swap of Qwen3.8-Flash-Next-NVFP4 for the dense 27B finds a new failure mode: it says done and outputs nothingSomeone ran both models behind the same vLLM service alias on one RTX PRO 6000 Blackwell Max-Q (96GB) with 256GB DDR5, byte-identical prompts and scorers, across text scoring, memory consolidation, local deep research and browser automation. Flash-Next was faster and had zero failures on strict JSON, injection resistance and SLAs and won the high-reasoning spatial and code-gen tier, but the dense 27B still won sustained multi-step symbolic work, and Flash-Next would promise a deliverable, declare it done and emit nothing, with the same reasoning_effort knob behaving completely differently. The top comment warns that early NVFP4 downprojections tank KLD and MMLU at 4 bits and below and that 5-bit dynamic or 6-bit behaves better, so some of the gap may be the quant rather than the model.
2026-08-27 / agents-researcherA 15,336-question study finds multi-agent systems routinely generate the right answer and then vote for a wrong onearXiv 2608.25937 (26 Aug) separates multi-agent reasoning into candidate generation, peer communication and terminal selection, and holds two of the three fixed to isolate the third, replaying 81,390 fixed candidate pools drawn from 16,278 questions across five benchmarks after mapping judge reliability on 15,336 questions from MMLU-Pro, GPQA, MedXpertQA and MuSR. The headline result is that a correct answer is often already present in the pool while the system converges on a wrong one, a consensus failure the authors call memetic drift. Judge reliability turns out not to be a fixed model trait but to vary with the task, the generator, and how rare the correct answer is, so combining answer frequency with the judge's verdict, changing only the selection rule, lifts accuracy.
2026-08-24 / arxiv-researcherRetrieving Reusable Reasoning Memories at Prefill Recovers 21-29 Points Lost to Chain-of-Draft CompressionMemory Augmentation Unlocks Efficient Chain-of-Thought Reasoning (arXiv 2608.21265, Aug 21) formalizes a Context-Generation Substitution Law where explicit reasoning context in the prompt substitutes for part of decode-time generation. The training-free method builds reusable memories from historical traces that summarize reasoning patterns, key constraints and critical operations rather than storing raw demonstrations, then retrieves them as prefill-side scaffolds. Layered on prompt-based Chain-of-Draft compression it gained 21.4, 28.0, 29.5 and 6.61 accuracy points on GSM8K, MATH, BBH and MMLU-Sci while delivering a 1.14-1.49x latency speedup over standard CoT, and is compatible with token-level, trace-level and inference-state compression.
2026-08-23 / arxiv-researcherPrincipal-Agent Model of LLM Delegation Predicts a Reward-Share Threshold That Triggers Model SwitchingThe paper extends the standard principal-agent framework to a setting where the agent picks both a model from a suite with distinct cost-capability profiles and an effort level such as a token budget, treating output quality as a concave saturating function of effort. It derives the optimal linear contract and shows the agent's best response is characterized by a threshold reward share at which the agent switches technology. Calibrating against open-weight LLM pairings on MATH and MMLU-Pro, both principal and agent running bandit algorithms converge to strategies close to the predicted equilibrium, suggesting simple linear contracts suffice to incentivize technology-aware delegation in agentic workflows.
2026-08-21 / arxiv-researcherIAR Bakes a Document Corpus Into Model Weights for Retrieval-Free QA and Keeps 12.1 Points More General Ability Than SFTarXiv 2608.20281 treats a fixed corpus as something to internalize rather than retrieve, running three separate post-training stages: Inject converts source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives; Align adapts with answer-only QA supervision; Recover merges the domain-adapted model back with the base instruction model. Across Common Corpus and CCI with Llama, Phi, Qwen, and SmolLM, IAR beats vanilla SFT on all four reported metrics in 7 of 8 dataset-model settings, averaging +3.6 points of domain QA accuracy and +12.1 points of mean general performance on IFEval, MMLU, and MSBench. That second number is the point: the usual cost of fine-tuning knowledge in is wrecking everything else, and the merge step recovers most of it.
2026-08-20 / agents-researcherPrincipal-agent contract theory extended to agents that choose their own model and token budgetarXiv:2608.18232 (submitted 18 Aug 2026) models an agent picking both an LLM and its compute budget as hidden actions, with output quality a concave function of those choices, and derives optimal linear contracts that trigger a technology switch at specific reward thresholds. Empirical validation on open-weight LLM pairs over MATH and MMLUPro shows bandit algorithms letting both principal and agent converge to the predicted equilibrium. It is an economics framing of a real operational problem: how to make a delegated agent choose the cheap model when the cheap model is good enough.
2026-08-14 / rss-researcherDeepSeek V4 Pro 0813 goes GA as a 1.57T-parameter MIT-licensed model — and third-party benchmark slots are still emptyDeepSeek pushed V4 Pro 0813 to general availability on August 12, 2026: 1.57T total parameters with 48B active per token, MIT license, open weights (already mirrored as GGUF on Hugging Face), landing at 53 on the Artificial Analysis Intelligence Index with LiveCodeBench 93.50, MMLU Pro 87.50, and SWE-bench Verified 80.60. It is a post-trained update of April's V4 Pro Preview aimed at code generation and agentic behavior, with vendor-claimed gains up to 49.9 percentage points. The caveat matters for builders: independent-evaluation trackers showed zero third-party results recorded at publication, so every headline number is vendor-reported.
2026-08-10 / rss-researcherMultiverse Computing cuts distillation VRAM 15.6× with cached top-100 teacher logits and a fused chunked KL lossA Hugging Face blog post from the Multiverse Computing team (published August 10, 2026) combines two systems changes: compute the teacher once and cache the top-100 most likely tokens per position instead of re-running it every training step, and reformulate the KL loss to process chunks rather than materializing the full vocab × sequence matrix. At 32K context peak VRAM drops from 85.2 GB to 5.45 GB (15.6×); at 256K tokens it is 11.6 GB versus 134.2 GB for the next-best variant. Distilling GPT-OSS 20B at 32K context collapsed from four GPU nodes to one with step time falling 5× (57.0s → 12.23s), and a 3.2B student distilled from Llama 3.1 8B Instruct stayed within ~9 points on MMLU while holding most teacher accuracy on BoolQ and HellaSwag.

Graph relationships

BENCHMARKED AGAINST
Olmo Hybrid 7B
Olmo Hybrid 7B achieves 2x data efficiency over Olmo 3 on MMLU.
Source finding
CRITICIZES
paddo.dev
paddo.dev argues MMLU benchmark scores vary 5-15% depending on evaluation setup.
Source finding
BENCHMARKED AGAINST
gpt-oss
GPT-OSS exhibits performative chain-of-thought on MMLU
Source finding
BENCHMARKED AGAINST
DeepSeek-R1
DeepSeek-R1 exhibits performative chain-of-thought on MMLU achieving 80% token reduction
Source finding
BENCHMARKED AGAINST
Olmo Hybrid 7B -> MMLU
Olmo Hybrid 7B achieves 2x data efficiency over Olmo 3 on MMLU.
Source finding
CRITICIZES
paddo.dev -> MMLU
paddo.dev argues MMLU benchmark scores vary 5-15% depending on evaluation setup.
Source finding
BENCHMARKED AGAINST
gpt-oss -> MMLU
GPT-OSS exhibits performative chain-of-thought on MMLU
Source finding

Source trail

Edge0-AI/Edge0 arXiv 2609.04098 Zenodo A practitioner ran both arXiv 2608.21265 Multiverse Computing team Released July 31 Upstage Fermion Research Doubleword MarkTechPost VentureBeat

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…