Tung Pham

Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models

Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham, Toan Tran

State-based fine-tuning has emerged as a compelling alternative to weight-based adaptation for transformers, updating lightweight controls into states rather than model weights, offering substantial memory savings while retaining parameter efficiency. However, most existing state…Read more