arxivcs.LG2026-07-07
Efficient Long-Horizon Learning for Learned Optimization
Xiaolong Huang, Benjamin Thérien, James Harrison, Eugene Belilovsky
Learned optimization aims to improve upon hand-designed optimizers (e.g., Adam and Muon) by meta-learning small neural network optimizers over a distribution of tasks. While recent work has greatly advanced the architectural design and inductive biases of learned optimizers (LOs)…