KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models

arXiv 2603.01875·medium signalXBlueskyLinkedInCopy link

Addresses the throughput bottleneck in knowledge distillation when the teacher is a Mixture-of-Experts model — baseline frameworks hit severe bottlenecks with training times exceeding 36 seconds per iteration for MoE teachers. KDFlow achieves 1.44x speedup by optimizing teacher execution during distillation, with the largest gains specifically for MoE-based teachers like Mixtral or Qwen-MoE variants. A practical engineering contribution for teams building smaller, specialized models by distilling from large MoE teachers.