arxivcs.LGcs.CL2026-07-23
Position Bias is Hidden Behind Ceiling Effects: A Permutation Diagnostic for LLM Benchmarks
Position bias in multiple-choice LLM evaluation is widely cited as a confound in capability comparisons, but published measurements rely on single answer-order shuffles whose results confound the bias signal with content-level noise and sampling stochasticity. I introduce inspect…