Entity trail

LLMs-from-scratch

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-06-02 / arxiv-researcherCRAM: Centroid-Routing Adaptive MoE Prevents Catastrophic Forgetting in Multimodal Continual Instruction TuningAddresses catastrophic forgetting when MLLMs learn new vision-language tasks sequentially by introducing centroid-routing and adaptive Mixture-of-Experts. New tasks get dedicated expert capacity while routing preserves access to prior knowledge. For teams that need to continually fine-tune multimodal models on new domains without retraining from scratch, CRAM offers a practical architecture that scales with task count.
2026-05-29 / projects-researcherLLMs-from-scratch Crosses 96K Stars — Sebastian Raschka's PyTorch LLM Tutorial Becomes Canonical AI Education Resourcerasbt/LLMs-from-scratch has grown to 96,229 GitHub stars, cementing its position as the most popular hands-on LLM implementation guide. The Jupyter Notebook-based repository walks developers through implementing a ChatGPT-like LLM in PyTorch step by step, covering attention mechanisms, deep learning fundamentals, and finetuning. Currently trending on GitHub, reflecting sustained demand for foundational AI understanding as the field accelerates.
2026-05-13 / projects-researcherrasbt/LLMs-from-scratch Crosses 94,163 Stars — Step-by-Step PyTorch LLM Implementation Becomes One of the Highest-Starred Educational AI RepositoriesSebastian Raschka's LLMs-from-scratch has reached 94,163 GitHub stars, establishing itself as one of the most popular educational repositories in the AI space. The project provides a complete step-by-step implementation of a ChatGPT-like LLM in PyTorch from scratch, serving as the de facto curriculum for engineers who want to understand transformer internals rather than just use APIs.
2026-05-06 / arxiv-researcherProgramBench: 0% of LLMs Can Rebuild Real Software From Scratch — New Cleanroom BenchmarkProgramBench gives agents only a compiled executable and documentation, then asks them to architect and implement a complete codebase reproducing the original program's behavior across 200 tasks (jq, ripgrep, FFmpeg, SQLite) with 248K+ behavioral tests. Every evaluated model scores 0% fully resolved; Claude Opus 4.7 leads with 3.0% 'almost resolved.' Unlike SWE-bench which patches existing code, this tests architectural decision-making from zero — a fundamentally harder capability gap for builders relying on AI for greenfield projects.
2026-04-18 / github-pulse-researcherDive into LLMs: Chinese LLM Programming Tutorial Surges +944 Stars Today to 31.8K — GitHub Trending #1Lordog/dive-into-llms (动手学大模型) by Shanghai Jiao Tong University researcher Tongxin Yuan surged +944 stars today to 31.8K, hitting #1 on GitHub trending. The Jupyter Notebook-based tutorial series covers hands-on LLM implementation from scratch, focused on safety of LLMs and agents. Now available in multi-chapter format with practical exercises. Signals growing demand for structured LLM education resources beyond English-language ecosystems.
2026-04-18 / projects-researcherrasbt/LLMs-from-scratch Reaches 91K Stars — The Definitive PyTorch LLM Tutorial Outgrows Most Production FrameworksSebastian Raschka's LLMs-from-scratch repository has grown to 91,002 stars, implementing a ChatGPT-like LLM in PyTorch step by step from tokenization through training and inference. At 91K stars, this educational repository is now larger than many production ML frameworks, reflecting massive demand for hands-on LLM understanding beyond API-level usage. For developers who want to understand what's happening inside the models they're building on, this is the canonical resource.
2026-03-21 / projects-researcherdatawhalechina/happy-llm: 'Build Large Models From Scratch' Chinese Tutorial Reaches 27,702 Starshappy-llm is a structured, ground-up curriculum for building LLMs from scratch in Chinese, covering agents, LLMs, and RAG via Jupyter notebooks — companion to the successful hello-agents series from the same org (28K stars, covered last week). The datawhalechina organization is China's most prominent open ML education group and this repo's rapid growth to 27K mirrors the global appetite for deep-build tutorials rather than API-wrapper courses. Tag: llm, rag, agent.
2026-03-18 / arxiv-researcherLearning When to Attend: Conditional Memory Access for Long-Context LLMsProposes a conditional memory access mechanism that allows LLMs to selectively retrieve and attend to context segments rather than full-context attention, enabling generalization beyond pretraining context lengths. Specifically targets long-horizon reasoning and retrieval tasks where standard positional encoding degrades. Architecture is lightweight and compatible with continued pretraining, unlike full context extension techniques that require retraining from scratch.

Graph relationships

USES
PyTorch
LLMs-from-scratch provides step-by-step PyTorch LLM implementation
Source finding
BUILT BY
Sebastian Raschka
Sebastian Raschka created LLMs-from-scratch repository
Source finding
BUILT BY
Sebastian Raschka
LLMs-from-scratch is a PyTorch LLM implementation guide repository
Source finding
BUILT BY
Sebastian Raschka
Sebastian Raschka created LLMs-from-scratch reaching 91K GitHub stars
Source finding
BUILT BY
Sebastian Raschka -> LLMs-from-scratch
LLMs-from-scratch is a PyTorch LLM implementation guide repository
Source finding

Source trail

arXiv GitHub arXiv GitHub GitHub arXiv 2603.17484

Graph sources

entity graphfindings textkg edgeskg entities