arxivcs.LG2026-07-09
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Yuanjie Zhu, Liangwei Yang, Ke Xu, Weizhi Zhang, Shanghao Li, Zihe Song, et al.
Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and t…