Three-Dimensional Autonomous Navigation of Unmanned Underwater Vehicle Based on Deep Reinforcement Learning and Adaptive Line-of-Sight Guidance

Jianya Yuan, Hongjian Wang, Bo Zhong, Chengfeng Li, Yutong Huang, Shaozheng Song

Unmanned underwater vehicles (UUVs) face significant challenges in achieving safe and efficient autonomous navigation in complex marine environments due to uncertain perception, dynamic obstacles, and nonlinear coupled motion control. This study proposes a hierarchical autonomous navigation framework that integrates improved particle swarm optimization (PSO) for 3D global route planning, and a deep deterministic policy gradient (DDPG) algorithm enhanced by noisy networks and proportional prioritized experience replay (PPER) for local collision avoidance. To address dynamic sideslip and current-induced deviations during execution, a novel 3D adaptive line-of-sight (ALOS) guidance method is developed, which decouples nonlinear motion in horizontal and vertical planes and ensures robust tracking. The global planner incorporates a multi-objective cost function that considers yaw and pitch adjustments, while the improved PSO employs nonlinearly synchronized adaptive weights to enhance convergence and avoid local minima. For local avoidance, the proposed DDPG framework incorporates a memory-enhanced state–action representation, GRU-based temporal processing, and stratified sample replay to enhance learning stability and exploration. Simulation results indicate that the proposed method reduces route length by 5.96% and planning time by 82.9% compared to baseline algorithms in dynamic scenarios, it achieves an up to 11% higher success rate and 10% better efficiency than SAC and standard DDPG. The 3D ALOS controller outperforms existing guidance strategies under time-varying currents, ensuring smoother tracking and reduced actuator effort.