Shubhanshu Shekhar

Efficient Sequential Evaluation of Large Language Models

Chia-Yu Hsu, Shubhanshu Shekhar

We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs. Our goal is to construct a confidence sequence (CS) for the model's capability on this question set and to design active que…Read more