Abstract We show that for unconstrained Deep Linear Discriminant Analysis (LDA) classifiers, maximum-likelihood training admits pathological solutions in which class means drift together, covariances collapse, and the learned representation becomes almost non-discriminative. Conversely, cross-entropy training yields excellent accuracy but decouples the head from the underlying generative model, leading to highly inconsistent parameter estimates. To reconcile generative structure with discriminative performance, we introduce the Discriminative Negative Log-Likelihood (DNLL) loss, which augments the LDA log-likelihood with a simple penalty on the mixture density. DNLL can be interpreted as standard LDA NLL plus a term that explicitly discourages regions where several classes are simultaneously likely. Deep LDA trained with DNLL produces clean, well-separated latent spaces, matches the test accuracy of softmax classifiers on synthetic data and standard image benchmarks, and yields substantially better calibrated predictive probabilities.
Abstract An AI assessor is an external, ideally independent system that predicts an indicator, e.g., a loss value, of another AI system. Assessors can leverage information from the test results of many other AI systems and have the flexibility of being trained on any loss functio…
Abstract In today’s digital landscape, user-generated opinions—such as online reviews, user comments, and social media posts—offer valuable insights into people’s experiences, sentiments, and concerns, influencing decisions across businesses, organizations, and public policy. Adv…
Abstract Simulating physical systems is essential in engineering, but analytical solutions are limited to straightforward problems. Consequently, numerical methods like the Finite Element Method (FEM) are widely used. However, the FEM becomes computationally expensive as problem…
Abstract While most continual learning methods focus on mitigating forgetting and improving accuracy, they often overlook the critical aspect of network calibration, despite its importance. Neural collapse, a phenomenon where last-layer features collapse to their class means, has…
Abstract Modeling the temporal evolution of biological systems is fundamental for understanding cellular dynamics and anticipating future functional states. While temporal graph neural networks (TGNNs) have achieved remarkable success in social and financial domains, their evalua…
Abstract Learning-based approaches have made substantial progress on solving combinatorial optimization problems, increasingly rivaling classical operations research methods. In particular, improvement-based machine learning methods, which iteratively refine an existing solution,…