Jiajun Shi

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang, Jiajun Shi

Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed labe…Read more