Wei-Min Huang

Backpropagation-Free Trunk Training via the Split Forward Gradients

Tian Qin, Wei-Min Huang

Backpropagation makes training deep networks memory intensive because it must store intermediate activations. Forward-mode methods avoid this cost, but their gradient estimates become increasingly noisy as the number of trained parameters grows. We introduce Split Forward Gradien…Read more