Bernd Rosenow

Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks

Marcel Kühn, Bernd Rosenow

The Hessian of the training loss governs the local geometry of the loss landscape, yet despite existing explanations for its largest eigenvalues, the origin of the vast multitude of vanishingly small eigenvalues remains elusive. We argue that the bulk consists of the weakly lifte…Read more