arxivcs.LGcs.CL2026-07-02
Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang, Geng Liu, Haiyang Guo, et al.
Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation emerging as a particularly attractive approach. In this work,…