Rohan Desai

FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts

Tom Saliencro, Maya Lindqvist, Rohan Desai, Priya Nair, Daniel Whitmore

Parameter-efficient fine-tuning (PEFT) reparameterizes weight updates in a fixed basis: low-rank adapters operate in the spatial domain, while a recent line of spectral methods operates in a fixed Fourier domain. We argue that the choice of domain is itself a design degree of fre…Read more