CORTEXA
← Browse
arxivcs.LGcs.AIcs.CLstat.ML2026-06-30

GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity

Yong Yi Bay, Kathleen A. Yearick

Three of the most popular methods for training language models to reason look like three different tricks. They are not. All three adjust a single number: standard deviation, reflecting how much a prompt's sampled answers disagree. When such a model is trained, it answers each problem many times, and an automatic checker marks every answer right or wrong. The standard deviation of those marks measures the disagreement: largest when the answers split evenly between right and wrong, and zero when they all agree. Group Relative Policy Optimization (GRPO) divides by this number, GRPO Done Right (Dr. GRPO) drops the division, and Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) discards the groups where it is zero. Each is presented as its own fix, yet this paper proves they are three settings of one dial. That dial is not cosmetic: for right-or-wrong rewards, the disagreement is exactly the size of the training update, the group-standard-deviation identity. A split group teaches the most, while a unanimous group teaches nothing and falls silent. The same result says which problems deserve the most weight and how many tries each one needs. This paper confirms the intuition on a large real difficulty dataset (Big-Math) and in a controlled training run. What looks like a harmless normalization step is the dial that decides where learning happens and how strongly.

View free PDFSource page

Related papers

arxivcs.LGcs.AIcs.CLstat.ML2026-06-27

When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling

Yong Yi Bay, Kathleen A. Yearick

People overthink; language models over-sample, and the extra effort can talk both into a worse answer. Reasoning systems answer a hard question by sampling it many times (test-time scaling), and the more they draw, the more often a correct answer turns up somewhere, so coverage,…

View free PDFSource page
arxivcs.LGcs.AIcs.CLcs.ITstat.ML2026-07-06

What Does a Discrete Diffusion Model Learn?

Rodrigo Casado Noguerales, Bernhard Schölkopf, Thomas Hofmann, Aran Raoufi

What does a discrete diffusion model learn: a denoiser, a score ratio, or a bridge plug-in predictor? At the level of jump rates, these are one object in different coordinates, and reading a neural network in the wrong coordinate changes the process being trained and sampled. Sta…

View free PDFSource page
arxivcs.CLcs.AIcs.LGstat.ML2026-07-05

CausalGame: Benchmarking Causal Thinking of LLM Agents in Games

Zhenhao Chen, Yongqiang Chen, Chenxi Liu, Junchi Yu, Xiangchen Song, Zijian Li, et al.

Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention. Since scientific discovery fundamentally relies on uncovering causal relationships from observations, the capability of causal thinking, i.e., distinguishing causation from co…

View free PDFSource page
arxivcs.LGcs.AIcs.CLstat.ML2026-07-06

Multi-Turn On-Policy Distillation with Prefix Replay

Baohao Liao, Hanze Dong, Christof Monz, Xinxing Xu, Li Dong, Furu Wei

We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn interaction histories. Fully online OPD is costly because each update requires fresh student rollou…

View free PDFSource page
arxivcs.CLcs.AIcs.LGstat.ML2026-07-05

Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability

Christopher Nassif, Josh F. Cooper

Distinguishing Large Language Model (LLM) generated text from human writing is a critical and difficult challenge. While LLMs are trained to write like humans, we hypothesize that this training leaves an indelible mark. LLMs develop a particularly strong aversion to token repetit…

View free PDFSource page
arxivcs.AIcs.CLcs.LGstat.APstat.ML2026-07-02

Online Safety Monitoring for LLMs

Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron, Eric Nalisnick

Despite alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an exter…

View free PDFSource page