CORTEXA
← Browse
arxivcs.LGcs.AIcs.CL2026-07-14

Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques

Daehoon Gwak, Minhyung Lee, Junwoo Park, Jaegul Choo

Diffusion large language models (dLLMs) offer a theoretical advantage in parallel generation over standard autoregressive models. However, parallel generation alone does not guarantee practical speedups. Realizing this efficiency requires specialized inference mechanisms, such as diffusion-aware caching and reuse. Consequently, as inference efficiency becomes a prerequisite for practical deployment, recent research has actively explored acceleration techniques across algorithms, architectures, and systems. However, rigorous comparisons remain difficult, as end-to-end latency stems from intricate trade-offs between algorithmic, architectural, and system-level factors that are often conflated in existing benchmarks. In this survey, we introduce a unified latency decomposition framework for dLLMs to disentangle these factors and analyze their impact on inference speed in real deployments. Guided by this framework, we categorize acceleration techniques along three axes covering algorithmic innovations, architectural and system optimizations, and inference-time scaling. Finally, we provide guidelines for reproducible benchmarking and highlight open challenges for realizing the full potential of parallel generation.

View free PDFSource page

Related papers

arxivcs.CLcs.AIcs.LG2026-07-12

Large Language Models for Token-Efficient and Semantic-Preserving Opinion Summarization

Fabrizio Marozzo, Stefano Iannicelli

Opinionated text - spanning product reviews, hotel feedback, and social posts - captures rich signals about user experiences, preferences, and concerns. However, the scale, redundancy, and imbalance of such corpora make it challenging to analyze opinions effectively, particularly…

View free PDFSource page
arxivcs.AIcs.CLcs.LGcs.SIphysics.soc-ph2026-07-13

Reproducing human biases in route choice using large language models: Toward scalable behavioral modeling

Jiangtao Han, Shoufeng Ma, Shuxian Xu, Geng Li, Shuai Ling, Ning Jia, et al.

Human choice behavior, including route choice, exhibits systematic behavioral biases that deviate from the assumptions of full rationality. Cumulative prospect theory (CPT) has been widely recognized as an effective framework for characterizing such behavioral patterns. However,…

View free PDFSource page
arxivcs.CLcs.AIcs.LG2026-07-16

Mask-Aware Policy Gradients for Diffusion Language Models

Haran Raajesh, Kulin Shah, Adam Klivans, Philipp Krähenbühl

Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation. Existing approaches approximate this log-like…

View free PDFSource page
arxivcs.CLcs.AIcs.LG2026-07-17

Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models

Andy Catruna, Emilian Radoi

While the internal mechanisms of autoregressive (AR) transformers have been studied extensively, much less is known about diffusion language models (DLMs), an emerging alternative that generates text by iterative denoising. In this work, we study how DLMs implement induction, a m…

View free PDFSource page
arxivcs.CLcs.AIcs.LG2026-07-18

Trace-Based On-Policy Distillation for Masked Diffusion Language Models

Haolin Ren, Ziyang Huang, Chenhao Yuan, Jun Zhao, Kang Liu

Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. However, reasoning-oriented post-training for dLLMs remains challenging. Supervised fine-tuning (SFT) for dLLMs requires dense but often off-policy masked states, while reinforcement…

View free PDFSource page
arxivcs.LGcs.AIcs.CL2026-07-01

CausalMix: Data Mixture as Causal Inference for Language Model Training

Zinan Tang, Yukun Zhang, Shaomian Zheng, Zhuoshi Pan, Qizhi Pei, Dingnan Jin, et al.

In Large Language Model (LLM) training, data mixing plays a pivotal role in determining model performance. Recent methods optimize mixture weights via proxy models, but they rely on the assumption of static data distributions. As a result, when the underlying data pool shifts, th…

View free PDFSource page