Dispatch
OpenArch hand-writes readable PyTorch implementations of 12+ modern LLM architectures
The repo has completed single-file implementations for GPT-2 XL, Llama 2/3/4 Maverick, OLMo 2, DeepSeek R1, Gemma 3, Mistral 3, Qwen 3, Kimi K2, GLM 4.5, GPT-OSS and PaliGemma, with Qwen3 multimodal in progress, each making attention type, normalization and positional encoding explicit. The stated goal is clarity over competing with transformers, targeting all 72 architectures in Sebastian Raschka's LLM Architecture Gallery. 106 stars and 101 commits, so it is early but already covers the models most people actually deploy.
↳ Follow the thread