Xiaoxuan Zhang, Zhenliang Xiong, Yinglin Zhou, Xianchun Zeng
Gemini 3 Pro showed promising discrimination in this selected retrospective cohort, but cross-model comparisons should be interpreted cautiously because inputs and score resolution differed. Prospective multicenter validation and broader natural language processing comparisons are required before clinical use. Predictive values may not transport to lower-prevalence populations, and estimates derive from a single historical inference run that may not be fully reproducible.
PURPOSE: Cognitive impairment in obstructive sleep apnea (OSA) may not be adequately captured by the apnea-hypopnea index (AHI). This study evaluated large language models (LLMs) for identifying Montreal Cognitive Assessment (MoCA)-defined cognitive risk from demographic, polysomnographic, and MRI reports.
METHODS: We retrospectively enrolled 111 adults with PSG-confirmed OSA. GPT-5.2 and Gemini 3 Pro were benchmarked against the AHI rule, structured logistic regression models, a gradient-boosted tree, and TF-IDF models. The primary outcome was MoCA < 26.
RESULTS: 73 participants (65.8%) met the primary MoCA-defined screening criterion. The AHI rule showed poor discrimination (AUC 0.456, 95% CI 0.383-0.530). AUCs were 0.759 (95% CI 0.660-0.846) for the age-and-education model, 0.749 (95% CI 0.650-0.841) for the full structured model, 0.736 (95% CI 0.637-0.827) for the gradient-boosted model, and 0.730 (95% CI 0.619-0.826) for the input-matched TF-IDF model. GPT-5.2 achieved an AUC of 0.698 (95% CI 0.584-0.805) with 98.6% sensitivity but 47.4% specificity. Gemini 3 Pro achieved the highest discrimination (AUC 0.884, 95% CI 0.822-0.934), exceeding the age-and-education model, the full structured model, the gradient-boosted structured model, and the input-matched TF-IDF model after Holm correction within their respective comparison families.
CONCLUSIONS: Gemini 3 Pro showed promising discrimination in this selected retrospective cohort, but cross-model comparisons should be interpreted cautiously because inputs and score resolution differed. Prospective multicenter validation and broader natural language processing comparisons are required before clinical use. Predictive values may not transport to lower-prevalence populations, and estimates derive from a single historical inference run that may not be fully reproducible.