SHADOW-50M: a 44M-parameter ternary LLM in 19.8 MB that hands arithmetic to a fixed readout circuit instead of a tool call
Follow-up to the author's SHADOW-250M, this model is 44M parameters trained from scratch on 45B tokens, shipping at 19.8 MB with ternary {-1,0,+1} weights, a 73,880-token vocabulary encoded as fixed 512-bit fingerprints rather than a learned embedding, a 159 KB compiled kernel, ~41 MB RSS and ~1,900 tok/s on laptop CPU (~500 tok/s in a browser via WebAssembly). The interesting mechanism is that when it decides something needs computing it emits `[calc]347*86[eq]` and a fixed circuit at the readout fills the digits inline, with no calculator API or tool round-trip; circuits exist for arithmetic, percentages, dates, weekdays, units, counting, sorting and comparisons. The top reply demonstrates heavy overfitting, the model returning the identical joke to two different prompts, and the author calls it a proof of concept.
↳ Follow the thread