Apple: Embarrassingly Simple Self-Distillation Improves Code Generation — No Labels, No RL, No Teacher

arXiv·medium signal

Apple researchers demonstrate that sampling solutions from a base model at temperature and fine-tuning on those raw unverified samples — requiring zero human labels, no teacher model, no reward model, no verifier, and no reinforcement learning — improves Qwen3-30B from 42.4% to 55.3% pass@1 on LiveCodeBench v6, with gains concentrating on hard problems. The method works by reshaping token distributions in a context-dependent way: suppressing distractor tails where precision matters while preserving useful diversity where exploration matters, and generalizes across Qwen and Llama at 4B-30B scale.