Lachlan MacDonald

SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates

Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun, Rene Vidal

Modern deep learning has been shown to operate at the edge of stability, routinely using learning rates far larger than those justified by classical optimization theory. Most prior analyses of the edge of stability phenomenon focus on deterministic gradient descent, leaving the s…Read more