Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives

Ruiqi Chen, Yibo Meng, Huidi Lu, Xiaolan Ding

Generative AI is increasingly used for everyday health guidance, yet its clinical appropriateness in chronic disease contexts remains poorly understood. This paper presents a two-part mixed-methods study on \revise{Type 2 Diabetes Mellitus (T2DM)}, examining how patients and physicians assess AI-generated health information. \revise{Study~1} analyzes 784 \revise{participant reported} patient queries to characterize seven informational need categories and \revise{develops a structured five dimensional physician rating rubric informed by patient query categories and clinician priorities} (\textit{Accuracy, Safety, Clarity, Integrity, Action Orientation}). \revise{Study~2} engages seven physicians scoring responses from four AI models and discussing evaluative reasoning through in-depth interviews. Models perform well on factual explanation and lifestyle guidance but consistently underperform on medication reasoning and emotional support. Two \revise{analytic concepts} emerge \revise{from the data}. The \textit{pre-visit primer} \revise{frames AI as preparation for clinical encounters rather than as a replacement for physicians}. The \textit{fluency illusion} \revise{describes how polished language may convey epistemic authority that the clinical content does not support}. Patients and physicians converged on three shared limitations (role boundaries, emotional inadequacy, personalization gaps) while diverging in evaluative emphasis, \revise{which informed} four design directions, task-aware orchestration, risk-aware fallback, dynamic personalization, and emotionally attuned interaction.