arxivcs.LGcs.AI2026-06-30
Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models
Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham, Toan Tran
State-based fine-tuning has emerged as a compelling alternative to weight-based adaptation for transformers, updating lightweight controls into states rather than model weights, offering substantial memory savings while retaining parameter efficiency. However, most existing state…