A Cardiac Surgery-Specific Artificial Intelligence Model to Automate Risk Stratification for 30-day Readmissions Using Discharge Summaries.

BACKGROUND: Readmission risk can be assessed with risk calculators but typically requires manual chart review. We aimed to develop an artificial intelligence (AI) large language model (LLM) to predict 30-day readmission risk by directly analyzing electronic discharge summaries of cardiac surgery patients. METHODS: We developed a cardiac surgery-specific LLM by fine-tuning a general medical LLM to predict 30-day all-cause readmissions directly from unaltered text in discharge summaries of 5244 adult cardiac surgery patients (2014-2021). Strict temporal validation was implemented in a test cohort (2021-2024; n = 3031). Model predictions were compared with readmission length of stay. RESULTS: The model achieved an area under the curve of 0.71, with sensitivities and specificities of 78.6% and 50.2%, respectively. Performance showed temporal degradation from 2022 to 2024 (area under the curve, 0.78 to 0.67), mitigated through recalibration. Patients in the highest vs lowest predicted risk score quintile had a greater mean readmission length of stay (7.1 vs 3.7 days; P < .001). CONCLUSIONS: A cardiac surgery-specific LLM showed high sensitivity, moderate specificity, and modest ability to discriminate 30-day readmission risk. Temporal trends in documentation influenced model performance, underscoring the need for dynamic recalibration to update AI models. This study suggests that AI-automated screening to assess readmission risk at the time of discharge is possible and may help inform readmission prevention strategies.