arxivcs.LG2026-07-09
Super Weights in LLMs and the Failure of Selective Training
Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag
Recent work identified Super Weights, individual parameters whose removal degrades model performance by orders of magnitude. We show that this degradation due to pruning Super Weights does not universally apply to all LLMs. Furthermore, if these parameters are so important, Super…