João F. Seabra

GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries

Miguel P. Bento, João F. Seabra

Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in-training by introducing a LogSumExp term to the loss that breaks the symmetries, thus selecting a basis that minimizes activation…Read more