Pranab Sahoo

LAARA: Layer-Aware Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

Ashutosh Tripathi, Surya Deep Singh, Pranab Sahoo, Sriparna Saha

Low-Rank Adaptation is widely used for parameter-efficient fine-tuning, yet existing methods typically assign the same adapter rank to every transformer layer despite their heterogeneous adaptation requirements. In this work, we show theoretically and empirically that uniform ran…Read more