Lachlan Ewen MacDonald

Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

Lachlan Ewen MacDonald, René Vidal

An important quantity in the theory of gradient descent (GD) is the \emph{sharpness}, defined as the largest eigenvalue of the objective Hessian. Classical analyses typically require the step size to be uniformly smaller than twice the reciprocal of the sharpness, but this condit…Read more