arxivcs.LGmath.OCstat.ML2026-07-04
A Structural Interpretation of GELU and Threshold-Transmission Activations via the First-Order Loss Function
The Gaussian Error Linear Unit is usually motivated as the expected output of an input-dependent Bernoulli gate. This work gives an alternative interpretation: GELU is the expected output of a hard linear gate with a Gaussian random threshold. This view provides a generative inte…