Zyphra Releases ZAYA1-8B: Reasoning MoE With <1B Active Parameters Matches DeepSeek-R1 on Math — Introduces Markovian RSA for Unbounded Reasoning at Constant Memory

Zyphra·medium signal

Zyphra released ZAYA1-8B, a MoE model with 8.4B total / 760M active parameters, trained on AMD Instinct MI300X. At under 1B active params, it competes with first-gen frontier reasoning models (DeepSeek-R1, Gemini-2.5-Pro) on AIME, HMMT, LiveCodeBench, and GPQA-Diamond. Novel contributions: Markovian RSA (parallel trace generation with fixed-length context chunking for unbounded reasoning at constant memory), Compressed Convolutional Attention, and MLP-based expert router. Apache 2.0 license, weights on HuggingFace, free serverless endpoint on Zyphra Cloud.