Amortized Inference for Sampling Distributions Where the Bootstrap Fails
Efron's bootstrap is the default tool for estimating the sampling distribution of a statistic, yet it is provably inconsistent for maxima of bounded-support distributions, means under infinite variance, extreme quantiles, and tail-index estimators. The classical remedies, the m-out-of-n bootstrap and subsampling, require rate corrections that depend on unknown parameters and behave erratically at realistic sample sizes. We propose an amortized alternative: a neural network is trained on simulated datasets drawn from a prior over a distribution family, using single independent draws of the root T_n - T(F) scored by the pinball loss, a proper scoring rule whose population minimizer is the posterior-predictive law of the root. At test time, a single forward pass maps one dataset of n = 200 observations to its full sampling-distribution estimate, from which confidence intervals follow directly. On four canonical bootstrap-failure problems (bounded-support maximum, alpha-stable mean, Pareto tail index, and 99% value-at-risk under tempered stable returns), the method attains nominal 95% coverage, beats every feasible classical method in Wasserstein distance to the true sampling distribution, and captures over 97% of the achievable improvement where the exact Bayes-optimal answer is computable. For the value-at-risk problem no distribution-free method can reach nominal coverage at all; the learned method attains 94.7%. A single universal network with a statistic token matches all four specialists, and on real daily market returns the unchanged model averages 0.87 coverage against 0.73 for the bootstrap, as predicted by our out-of-family analysis.