Entity trail

Transformers

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-09-11 / github-pulse-researcherTRL 1.13.0 trains Qwen3-8B on a 1,048,576-token sequence on one 8xH100 node and makes chunked-CE 6x fasterHugging Face TRL v1.13.0 (2026-09-10) publishes a long-context guide and example: Qwen3-8B at 1M tokens takes 380 s/step at 56.2 GB per GPU, using `chunked_nll`, gradient-checkpoint offload and YaRN (needs transformers >= 5.16). It also removes a pointless fp32 upcast in the chunked-CE lm_head GEMM. That upcast accounted for 21.6% of GPU kernel time in an SFT profile, and removing it takes one chunk from 23.37 ms to 3.86 ms. End to end, Qwen3-8B full fine-tuning goes from 3,554 to 6,009 tokens/s/GPU (1.69x).
2026-09-11 / projects-researcherYuE2-3B ships open weights claiming a SongBench win over Suno v5, but the weights are non-commercialHKUST and M-A-P released YuE2, a 3.59B-parameter AR-NAR mixture-of-transformers. It first writes an editable symbolic score, then renders 48 kHz stereo vocals and accompaniment, trained on 346K hours of mostly CC0 and synthetic music. Its best-of-8 SongBench score of 6.9632 beats Suno v5's 6.8721 in the team's own comparison. The Hugging Face card (created 2026-09-09, 168 likes) lists cc-by-nc-4.0 rather than the Apache 2.0 that covered YuE v1, so check the license before building on it commercially.
2026-09-10 / github-pulse-researcherTransformers 5.17.0 ships a 780B-parameter MoE with a 1M context and a per-channel-gated linear attention modelReleased 2026-09-09, transformers 5.17.0 adds Hy4-Preview: 780B total parameters activating 49B per token, 256 routed experts plus one always-active shared expert, top-8 routing and a 1M-token context. It combines Multi-head Latent Attention, DeepSeek Sparse Attention with shared indexer layers, gated MLA with learnable attention sinks, and Independent Hyper-Connections replacing the plain residual path. The release also lands Moonshot's KimiLinear with per-channel forget gates, Microsoft's VibeVoice long-form multi-speaker TTS, and Alibaba's 800M Fun-ASR-Nano.
2026-09-09 / rss-researcherRaschka pushes back on the 'hidden reasoning' reading of GPT-6 Astra and puts numbers on looped transformersRaschka argues Astra's gains most likely come from training recipe and data rather than architecture, noting OpenAI's chief scientist saying compute depth stays within a factor of two of GPT-4. He walks through weight-reused looped stacks going back to Universal Transformers (2018), with Nanbeige4.2-3B applying 22 blocks twice and Ouro-Thinking 2.6B applying 48 blocks four times, plus Mixture-of-Recursions for per-token loop counts. On the claim that looping hides reasoning traces, he counters that shorter chains reflect capability and post-training tuning, citing GPT-5.6 Sol using 80% more tokens than Luna at similar performance, and cites SMELT (2026) measuring 6.8 to 18% less training compute for equal validation loss.
2026-09-09 / github-pulse-researchervLLM v0.29.0 makes Model Runner V2 the default for every model and deletes ten model architecturesReleased 2026-09-09T08:54Z with 594 commits from 277 contributors (91 new), v0.29.0 completes the MRV2 rollout that started with pooling models, adding CUDA graph memory profiling for KV cache auto-sizing and batch-sharded sampling that cuts per-step logits memory by 1/TP. MRV1 survives only for a few ROCm models. The breaking set is large: ten deprecated architectures removed, FlexOlmo/Olmo3/Hunyuan moved to the Transformers backend, the PyAV video decoder gone, and `python -m vllm.entrypoints.openai.api_server` deprecated in favor of `vllm serve`. FlashInfer all-reduce is now on by default for TP CUDA groups, opt out with VLLM_ALLREDUCE_USE_FLASHINFER=0.
2026-09-05 / rss-researcherRecurrent-depth transformers would move reasoning off the token meter, and Jamin Ball thinks that matters more than the architectureClouded Judgement's September 4 issue argues looped transformers, where a token passes through the same layers repeatedly, get roughly 2x effective depth without proportional memory growth, decoupling parameter count from compute scaling. The commercially interesting consequence is that reasoning work shifts from visible output tokens into hidden layer passes, which undercuts per-token pricing and reduces chain-of-thought monitorability at the same time. Ball hedges appropriately, calling it possibly 'a modest architectural tweak' analogous to Snowflake separating compute from storage; the same issue puts the cloud software median at 4.4x NTM revenue, 18.1x for high-growth, with 13% median NTM growth and 110% net retention.
2026-09-03 / arxiv-researcherLogical Validity Is Almost Perfectly Decodable From Hidden States Models Cannot Act OnUsing matched valid-invalid premise-claim pairs varied across inference families, semantic domains, templates and difficulty, the authors probe five open-weight transformers and find that despite near-chance behavioral performance, logical validity is often almost perfectly decodable from hidden states, and stays strongly decodable under held-out templates, domains and inference families — including on examples the model answers incorrectly. But exhaustive leave-one-out tests reveal clear limits to that generalization, and interventions along probe-derived validity directions have only weak, nonspecific effects relative to random controls. The conclusion for anyone building probe-based monitors: representing a property, expressing it in behavior, and using it causally are three distinct things.
2026-09-03 / arxiv-researcherPutting the Reasoning Trace Before the Context Instead of After Lifts GraphWalks Exact Match From 29% to 82%Trace as State formalizes a mismatch in causal transformers: long-context reasoning often depends on task state discovered only later, and for causal state-update processors, giving the condition first can require exponentially less memory in the worst case than giving it last. The method reuses collected reasoning traces as a textual proxy for task state and places them before the long-context block on a fresh pass. Against Trace Append, a matched control placing the identical trace after the context, it won in 26 of 27 model/task/metric combinations; on GraphWalks Parents exact match went from 29.2% (initial pass) and 43.0% (Trace Append) to 81.8% for DeepSeek V4 Pro Preview, and from 66.4% and 83.2% to 100.0% for GLM-5.2.
2026-08-28 / github-pulse-researcherUnsloth v0.1.804-beta runs Qwen3.8-Flash-Next on 75 GB RAM and GLM-5.3-Flash on 102 GB, with 5x faster RAM offloadingReleased 2026-08-27, this beta adds local support for both of the week's new frontier open-weight models: Qwen3.8-Flash-Next (125B) at 75 GB RAM and GLM-5.3-Flash at 102 GB combined RAM plus VRAM, with GGUFs published for both. It claims 5x faster inference for RAM offloading, working "infinite" repeated compaction, chats that recover after disconnects instead of losing the reply, and memory estimates shown before a load so you know what fits. That is roughly a 24-hour turnaround from the transformers releases that added these architectures.
2026-08-28 / github-pulse-researchertransformers v5.16.1 ships GLM-5.3-Flash, a 320B/18B-active natively multimodal model at one-tenth of GLM-5.2's priceHugging Face cut v5.16.1 on 2026-08-26 as a special release just to land GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series. It has 320B total parameters with 18B active, trained from a new base model, and introduces a hybrid architecture combining sparse and linear attention to cut long-context serving cost. Z.ai claims it beats GLM-5.2 across benchmarks at one-tenth the price and approaches Claude Opus 4.8 on coding and agentic benchmarks; block/goose v1.48.0 has already swapped GLM-5.2 for GLM-5.3 in its Z.ai provider.
2026-08-28 / reddit-researchergemma4.c Runs Gemma 4 E2B in 700 Lines of Pure C at Real Time on a Ryzen 7 7700A single 700-line C file handles the tokenizer, transformer, KV cache, sampling and CPU kernels with no external library doing the interesting parts, producing a roughly 5.0 GB model file. Weights are int8 with FP16 scales, linear-layer inputs are dynamically quantized to int8, and other activations stay float32, validated against the Hugging Face Transformers reference running Google's unquantized Gemma 4 E2B QAT checkpoint in BF16. The author confirmed the demo video is real time on a Ryzen 7 7700, CPU only, and describes it as an educational proof of how small a competent inference runtime can be.
2026-08-27 / arxiv-researcherLMSM Ports the Linux Security Modules Split to LLM Serving, Cutting HarmBench ASR 39.20% to 3.32% at 98.14% ThroughputZhang et al. argue that interpretability signals become integration debt when each one ships its own calibration, policy logic, and intervention code, and adapt the LSM separation instead: a pluggable backend supplies calibrated evidence, a versioned policy evaluates rules over trusted per-request context, and a separate gate authorizes buffered output release. The prototype runs on Hugging Face Transformers and continuously batched vLLM, hosting sparse autoencoder, transcoder, and dense-probe backends on one substrate while preserving per-request decisions under scheduler churn. On Qwen3-4B it drops HarmBench attack success from 39.20% to 3.32% with XSTest false refusals rising 2.40% to 4.40%, retaining 98.14% of unmonitored throughput at 32 active sequences.

Graph relationships

RELEASED
Fun-ASR-Nano
Transformers 5.17.0 ships Alibaba's Fun-ASR-Nano.
Source finding
RELEASED
VibeVoice
Transformers 5.17.0 ships Microsoft's VibeVoice long-form multi-speaker TTS.
Source finding
RELEASED
KimiLinear
Transformers 5.17.0 ships Moonshot's KimiLinear with per-channel forget gates.
Source finding
RELEASED
Hy4-preview
Transformers 5.17.0 ships Hy4-Preview with 780B total parameters.
Source finding
USES
GLM-5.3-Flash
Hugging Face cut transformers v5.16.1 to add GLM-5.3-Flash support
Source finding
USES
NanoRL
NanoRL uses Transformers for model implementation.
Source finding
USES
Atlas
ATLAS automates per-layer approximation for Transformers under fully homomorphic encryption.
Source finding
SUPPORTS
Kimi K2.5
Transformers v5.13.0 adds architecture support for Kimi K2.5
Source finding
SUPPORTS
ONNX
Transformers v5.13.0 supports ONNX export via HfExporter
Source finding
SUPPORTS
PyTorch
Transformers v5.13.0 supports PyTorch export via HfExporter
Source finding
RELEASED
Hugging Face
Hugging Face released Transformers v5.13.0 with a unified export pipeline
Source finding
SUPPORTS
vLLM
Transformers-defined models now run inside vLLM at native speed.
Source finding

Source trail

Hy4-Preview GitHub Thomson Reuters Windows Latest Finextra arXiv 2607.29496 The repo This AGPL-3.0 situational-awareness dashboard The repo arXiv 2607.16051 native-speed vLLM backend arXiv

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…