Embarrassingly Simple Self-Distillation Boosts Qwen3-30B Code Gen from 42.4% to 55.3% on LiveCodeBench

arXiv·high signal

Simple Self-Distillation (SSD) improves code generation using only the model's own raw outputs — no verifier, teacher model, or RL required. Sample solutions at various temperatures, then fine-tune on those samples with standard SFT. Qwen3-30B-Instruct jumps from 42.4% to 55.3% pass@1 on LiveCodeBench v6, with gains concentrating on harder problems. Technique is model-agnostic and trivially cheap to implement.