Tom Jacobs

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

Shuang Liang, Tom Jacobs, Guido Montúfar

We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression. In a mean-field regime, the training dynamics are approximated by a Wasserstein gradient flow that converges to a unique stationary measure. We ch…Read more