Qiang Tang

Single-Teacher View Augmentation: Enhancing Knowledge Distillation with Student-Guided Perturbations

Xuyi Yu, Yaohua Liu, Chengjun Li, Qiang Tang, Shuzhe Tang, Kuizhi Mei

Knowledge distillation (KD) typically relies on the fixed perspective of a single teacher, limiting the diversity of supervisory signals. While multi-teacher distillation addresses this by aggregating knowledge from multiple models, it incurs prohibitive computational and storage…Read more