Entity trail

LiveCodeBench

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / arxiv-researcherTest Case Scaling Trains a Test Generator Adversarially Against the Solver's Current Failure ModesRL for code generation depends on test cases that are both sound and discriminative, and those are scarce. TCS reframes test generation as an adversarial RL problem where the generator must produce counterexamples targeting the solver's present failure modes, training from a rolling policy-aligned buffer in two stages: stage 1 generates tests consistent with the reference solution, stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench it improves both pass@1 and inference-time answer selection, and the learned generator also works for selecting among outputs from other LLMs.
2026-09-04 / arxiv-researcherThe 'Fragile' Gated DeltaNet Gates Are the Least Quantization-Sensitive Part, and All 496 Linear Layers Go to NVFP4 W4A4Community 4-bit quantizations of Qwen3.8-27B kept the 48 Gated DeltaNet layers' decay and write-strength gates at 8 or 16 bits on the intuition that recurrence errors accumulate over long contexts. Minima tests that intuition by pushing NVFP4 W4A4 through all 496 linear layers including GDN, and matches BF16 within seed noise across 4K/32K perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER to 64K, with a 5-task average of -0.52, the smallest footprint at 17.5 GiB, and 14-19% faster prefill. The mechanism study finds the gate projections compress roughly 11% GEMM error to about 2% output error and the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens because each write overwrites state along the current key direction.
2026-08-31 / reddit-researcherAn "Agentic Coding Index" aggregating seven coding benchmarks lands 142 upvotes and an immediate methodology objectionThe index blends DeepSWE v1.1 (20%), Code Arena Elo (20%), Terminal-Bench v4.0 (15%), SWE-bench Pro (15%), Terminal-Bench v3.0 (13%), Terminal-Bench v2.1 (12%) and LiveCodeBench v6 (5%), then divides by sqrt(param count + an 8B regularization floor) raised to a 2.5354 super-linear exponent so tiny models cannot game the density score. The top comment asks the obvious question, how you get parameter counts for closed frontier models, and the answer is that API price is used as a proxy, which produces estimates like Claude Fable 5 at roughly 7.5T parameters from its $20/1M pricing. Useful as a shape, not as a number, since the denominator for every closed model is a guess derived from a business decision.
2026-08-31 / sources-researcherPLVR post-trains by learning an inspectable program instead of updating weightsPosted 2026-08-28, Program Learning with Verifiable Rewards argues that for tasks whose intermediate steps can be verified, reasoning belongs outside the weights as an explicit program of deterministic and neural primitives, because SFT and RL both bury the capability where it cannot be inspected, checked step by step, or transferred to another model. The mechanism is symbolic backpropagation: each program layer carries a typed ontology, loss is computed at the output, and required input ontologies propagate backward by type inference over primitive signatures, so credit assignment is a derivation rather than an estimate. Where RLVR verifies only a terminal outcome, PLVR's reward is a per-step contract verdict, evaluated on LiveCodeBench v6 and Tau2Bench.
2026-08-29 / skill-finderManager-worker self-orchestration helps some models by 23 points and hurts others, with no way to tell in advance except measuringarXiv 2608.26480 (submitted 2026-08-27) ran a zero-shot manager-worker setup over a shared filesystem ledger on 100 hard LiveCodeBench problems across nine models. Qwen3.8-27B gained 23.4 points, GPT-5.6-Luna 10.6, GPT-5.6-Terra 8.0, while Qwen3.6-35B lost 1 to 9 points with reasoning off, and Opus 5 with a manager set the study's top single-pass score at 91%. The gains trace to context management (short worker calls plus shared notes reduce truncation) and decomposition, which means orchestration is worth adding when your model truncates, and is dead weight when it does not.
2026-08-29 / reddit-researcherNew Qwen3.8-27B GGUFs Claim BF16-Level AIME25 at 2.75 bpw by Learning the Quantizer and the Search TogetherA release post on r/LocalLLaMA (98 upvotes) publishes Qwen3.8-27B GGUFs built with two methods: GSQ (Gumbel-Softmax Quantization), post-training scalar quantization that jointly learns grid assignments and scales at 2-3 bits, and RCO (Riemannian Constrained Optimization), which assigns a quant type per tensor under a strict size budget by gradient descent on the task loss. The 3.00 bpw build (10.1 GB) matches the BF16 base on AIME25 at 100.00, lands within about a point on GPQA-Diamond (88.89 vs 89.90) and LiveCodeBench v6 (84.57 vs 85.71), and the 2.75 bpw build (9.3 GB) still hits AIME25 100.00 with a zero-shot average exceeding BF16. All three files run unmodified in llama.cpp, Ollama and LM Studio.
2026-08-28 / agents-researcherA manager-worker scaffold beats a larger model on cost, but the gain is null or negative for a third of models testedA 27 August study isolates the manager-worker scaffold over a shared filesystem workspace against the same model answering in a single pass, on the 100 latest hard LiveCodeBench problems across nine models from 9B to about 2.8T parameters. The scaffold gives large significant gains for some (Kimi-K3 +30.4, Qwen3.8-27B +23.4, GPT-5.6-Luna +10.6) and is null to negative for others (Qwen3.6-35B -1 to -9 with reasoning off). Running a manager roughly triples the token bill but buys accuracy more cheaply than moving to a larger model, and Opus-5 with the manager posts the study's top score at 91% in one pass.
2026-08-23 / reddit-researcherThree Days of Independent Benchmarking on the DFlash 2 llama.cpp PR: 2.26x on Real Coding Prompts, and Two Documented Flags That Do NothingInco AI shipped DFlash 2 on 2026-08-18 with a Qwen3.8-27B drafter and llama.cpp PR #27342. A practitioner built the PR on an RTX PRO 6000 and measured 2.26x on 100 LiveCodeBench problems (67.97 to 153.91 tok/s, inter-token latency 14.27 to 6.02 ms) for +2.7 GB VRAM, and 4.68x on an 18-turn coding session with one n-gram lookup table, though adding the second table dropped it to 3.77x, reversing the DFlash 1 result. Two specific corrections to the docs: --spec-draft-n-max 7 is past the peak and 5 gives roughly 11% more on 8K prompts, and --spec-draft-p-min is never read on the DFlash 2 path in common/speculative.cpp.
2026-08-20 / skill-finderThree agents with structured disagreement beat a five-agent review panel on LiveCodeBenchAdversarial Review puts a reviewer and a critic around the main coding agent, where the critic audits the review through structured, evidence-grounded disagreement before any edit lands. It took the highest pass rate on LiveCodeBench while using three agents against a five-agent baseline, and the highest F1 on SWE-PRBench once an explicit disagreement prompt iteration was added. The conclusion for anyone building review pipelines is that the win comes from making disagreement minimal, structured, and evidence-grounded, not from adding agents.
2026-08-18 / arxiv-researcherTDD-Agent Turns Generated Tests Into Evolving Reasoning Artifacts Instead of Post-Hoc ValidatorsRather than using LLM-generated tests as static checkers — which misleads when the tests are themselves wrong — TDD-Agent prompts for executable tests first to force behavioral clarification, then runs iterative dual-track refinement over code and tests using execution feedback. A prompt-only variant (TDD-prompt) isolates the test-first effect and consistently improves on reasoning-based prompting baselines on LiveCodeBench; the full framework outperforms retrieval- and agent-based baselines on the repository-level RepoEval. Notably, refinement improves the tests too — higher pass rates, coverage and mutation scores — which is the part that transfers to real harness design.
2026-08-16 / rss-researcherQwen ships Qwen3.8-27B under Apache 2.0 — a 27.8B dense multimodal model claiming 61.7% on SWE-Bench ProAlibaba's Tongyi Lab released Qwen3.8-27B on 2026-08-14: 27.78B dense parameters, text/image/video input, Apache 2.0, and a native 262,144-token context extensible to 1M via YaRN. Reported scores include SWE-Bench Pro 61.7%, LiveCodeBench 90.3%, GPQA Diamond 89.2%, and Terminal-Bench 2.1 at 73.0 (up from 63.4 in Qwen3.6-27B), with DeepSWE 1.1 jumping 13.3 → 42.2. It arrived 11 days after Qwen3.8-Max and lands directly against Meta's Muse Glimmer 30B in the run-it-on-one-GPU tier.
2026-08-14 / rss-researcherDeepSeek V4 Pro 0813 goes GA as a 1.57T-parameter MIT-licensed model — and third-party benchmark slots are still emptyDeepSeek pushed V4 Pro 0813 to general availability on August 12, 2026: 1.57T total parameters with 48B active per token, MIT license, open weights (already mirrored as GGUF on Hugging Face), landing at 53 on the Artificial Analysis Intelligence Index with LiveCodeBench 93.50, MMLU Pro 87.50, and SWE-bench Verified 80.60. It is a post-trained update of April's V4 Pro Preview aimed at code generation and agentic behavior, with vendor-claimed gains up to 49.9 percentage points. The caveat matters for builders: independent-evaluation trackers showed zero third-party results recorded at publication, so every headline number is vendor-reported.

Graph relationships

BENCHMARKED AGAINST
Test Case Scaling
Test Case Scaling was evaluated on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
Opus 5
Opus 5 with a manager set the study's top single-pass score on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
Qwen3.8-27B
Qwen3.8-27B gained 23.4 points on manager-worker self-orchestration on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
Opus 5
Opus-5 with manager-worker scaffold achieved the highest score of 91% on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
Qwen3.8-27B
Qwen3.8-27B showed a 23.4% improvement on LiveCodeBench with manager-worker scaffold.
Source finding
BENCHMARKED AGAINST
Kimi K3
Kimi-K3 showed a 30.4% improvement on LiveCodeBench with manager-worker scaffold.
Source finding
BENCHMARKED AGAINST
DFlash 2
DFlash 2 achieved 2.26x speedup on 100 LiveCodeBench coding problems.
Source finding
BENCHMARKED AGAINST
Adversarial Review
Adversarial Review achieves the highest pass rate on LiveCodeBench with three agents.
Source finding
BENCHMARKED AGAINST
Solar Open 2
Solar Open 2 scored 92.4 on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
Qwen3-30B
Qwen3-30B-Instruct reaches 55.3% pass@1 on LiveCodeBench v6 with Simple Self-Distillation.
Source finding
BENCHMARKED AGAINST
Qwen3-30B
Qwen3-30B improved from 42.4% to 55.3% on LiveCodeBench v6.
Source finding
BENCHMARKED AGAINST
Test Case Scaling -> LiveCodeBench
Test Case Scaling was evaluated on LiveCodeBench.
Source finding

Source trail

arXiv 2609.04098 r/LocalLLaMA Level1Techs writeup llama.cpp PR #27342 AI Release Tracker Per Willison r/ClaudeAI A July 28 study Upstage MarkTechPost VentureBeat Presenc AI

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…