Jian Qian

Distributionally Robust Listwise Preference Optimization

Xudong Wu, Jian Qian, Pangpang Liu, Vaneet Aggarwal, Jiayu Chen

Existing robust preference optimization for language-model alignment mainly studies pairwise supervision and places robustness at the dataset, prompt, or preference-pair level. We instead study listwise preference optimization under ranking-label uncertainty: given a prompt and a…Read more