Study Maps When Training-Free Relaxed Speculative Decoding Actually Pays Off

arXiv·medium signal

Standard speculative decoding is lossless — its rejection/resampling steps exactly preserve the LLM's sampling distribution. This paper practically investigates relaxing that guarantee without any training, quantifying the speed-ups and controlled capability-speed trade-offs (and claimed capability gains) that relaxation can buy. Useful for practitioners tuning inference latency where a small distributional drift is acceptable.