HCCS: Clipped-Linear Softmax Surrogate Eliminates Exponentiation Bottleneck for Integer-Native Edge Inference

arXiv·medium signal

Head-Calibrated Clipped-Linear Softmax (HCCS) replaces the exponential softmax function with a bounded, monotone clipped-linear mapping that produces stable probability distributions without exponentiation or normalization. Targets small Transformer models under low-precision inference where softmax becomes a computational bottleneck in multi-head attention. For edge deployment of small language models, this removes a key overhead without requiring model retraining.