Machine Learning-Enhanced Last-Mile Delivery Optimization: Integrating Deep Reinforcement Learning with Queueing Theory for Dynamic Vehicle Routing

Tsai-Hsin Jiang, Yung-Chia Chang

We present the ML-CALMO framework, which integrates machine learning with queueing theory for last-mile delivery optimization under dynamic conditions. The system combines Long Short-Term Memory (LSTM) demand forecasting, Convolutional Neural Network (CNN) traffic prediction, and Deep Q-Network (DQN)-based routing with theoretical stability guarantees. Evaluation on modern benchmarks, including the 2022 Multi-Depot Dynamic VRP with Stochastic Road Capacity (MDDVRPSRC) dataset and real-world compatible data from OSMnx-based spatial extraction, demonstrates measurable improvements: 18.5% reduction in delivery time and +8.9 pp (≈12.2% relative) gain in service efficiency compared to current state-of-the-art methods, with statistical significance (p < 0.01). Critical limitations include (1) computational requirements that necessitate mid-range GPU hardware, (2) performance degradation under rapid parameter changes (drift rate > 0.5/min), and (3) validation limited to simulation environments. The framework provides a foundation for integrating predictive machine learning with operational guarantees, though field deployment requires addressing identified scalability and robustness constraints. All code, data, and experimental configurations are publicly available for reproducibility.