Principled approaches for extending neural architectures to function spaces for operator learning

Julius Berner, Miguel Liu-Schiaffini, Jean Kossaifi, Valentin Duruisseaux, Boris Bonev, Kamyar Azizzadenesheli, Anima Anandkumar

Abstract Deep learning has achieved remarkable success in computer vision and natural language processing, where tasks are commonly formulated as mappings between finite-dimensional representations. Many scientific problems, however, including those governed by partial differential equations, are naturally posed on infinite-dimensional function spaces. This mismatch has limited conventional neural networks from achieving comparable success in scientific applications. Here we identify and distil key principles for constructing practical neural architectures for mappings between function spaces. Neural operators provide a principled extension of neural networks to such settings, offering a path towards bringing deep learning’s transformative impact to science. Because deep learning’s success has relied heavily on architectural refinements, extending these advances to neural operators allows operator learning to benefit from refined designs. Guided by the principles we outline, we propose a recipe for converting popular neural architectures into neural operators with minimal modifications. We also discuss practical steps for making these models effective. This perspective offers a systematic bridge between finite-dimensional network design and operator learning for scientific applications.