Gate speculative decoding on batch size or it'll make you slower.

Speculative decoding (a small draft model proposes tokens the target verifies in parallel) gives 2-5x latency wins, but only in memory-bound, low-batch regimes. At large batch sizes the GPU is already compute-bound, and the extra draft-and-verify work makes inference slower than plain decoding (PremAI breakdown). SGLang's EAGLE implementation handles this adaptively, tracking an EMA of accepted draft length and switching speculative-length tiers with pre-captured CUDA graphs. The rule: don't leave it always-on.