arxivcs.CRcs.DScs.LG2026-07-08
Is Randomness Necessary for Adaptive Data Analysis?
Edith Cohen, Haim Kaplan, Yishay Mansour, Shay Sapir, Uri Stemmer
The Adaptive Data Analysis (ADA) problem formalizes the challenge of preventing false discovery and overfitting when a dataset is repeatedly reused. Formally, our input is a dataset containing $n$ i.i.d.\ samples from an unknown distribution $P$ over a domain $X$, and our goal is…