Vu Nguyen

Online Data Selection for Instruction Tuning via Gaussian Processes

Jun Wang, Quoc Phong Nguyen, Julien Monteil, Vu Nguyen

With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting o…Read more