Who Responds When the Driver Is Gone? A Framework for Human Intent Understanding

Xuewen Luo, Ding Fan, Ruiqi Chen, Ye Cao, Xiujin Liu, Bo Yu, Fengze Yang, Chenxi Liu

As autonomous vehicles progress toward fully driverless mobility, a critical question emerges: who understands and responds to passengers when the human driver is absent? Existing autonomous driving systems primarily optimize predefined navigation and control objectives from external scene observations, but they remain limited in perceiving and reasoning about in-cabin human intent. In this paper, we propose Intent2Drive, a unified framework for holistic human intent understanding and human-aligned planning. Instead of treating passenger intent as explicit commands alone, Intent2Drive models intent as a latent cognitive state shaped by language, personal attributes, emotional and physical conditions, behavioral signals, and situational context. To support this formulation, we construct a Holistic Intent Dataset (HID) that provides structured supervision over both explicit and implicit intent cues. Built upon HID, our Theory-of-Mind-inspired Human Intent Reasoner (HIR) infers a Latent Human State (LHS) and further translates it into a planner-compatible Human Intent Objective (HIO). We then introduce a Hierarchical Intent-Conditioned Planner (HICP) that incorporates HIO into route-level and trajectory-level planning, enabling driving behaviors to remain aligned with passenger needs across different planning horizons. Extensive experiments show that Intent2Drive improves structured human intent inference and HIO construction while preserving competitive closed-loop planning performance. These results demonstrate a promising step toward passenger-responsive autonomous driving systems that can reason about, interpret, and act upon human intent in driverless mobility.