Satwik Bathula

Fisher Rank Inflation: A Spectral Signature of Memorization under Label Noise

Satwik Bathula, Anand A. Joshi

Deep networks trained with label noise often learn clean structure before memorizing corrupted labels. We show that this transition leaves a spectral signature in the centered scatter of per-example last-layer gradients. Its effective rank transiently expands during memorization…Read more