Early Machine Learning-Based Identification of Hospitalized Patients at Low Risk of Respiratory Deterioration or Mortality in Community-Acquired Pneumonia: External Validation of a Multivariable Model.

BACKGROUND/OBJECTIVE: To externally validate our previously published machine learning model for identifying hospitalized patients with community-acquired pneumonia (CAP) at low risk of respiratory deterioration or death using the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. METHODS: This is a retrospective cohort study of adult patients (≥18 years) who were admitted with CAP and acute hypoxemic respiratory failure using the publicly available MIMIC-IV critical care dataset (Beth Israel Deaconess Medical Center, Boston, MA). We conducted an external validation study of a previously developed gradient boosting machine (GBM) model without recalibration using data available within the first 6 h of hospital admission. RESULTS: For the primary composite outcome (need for advanced respiratory support [high flow nasal cannula (HFNC), non-invasive mechanical ventilation (NIMV), invasive mechanical ventilation (IMV)] or in-hospital death), the gradient boosting model demonstrated comparable performance in the derivation and external validation cohorts. The area under the receiver operating characteristic curve (AUC) was 0.713 in the Mayo cohort (n = 4379) and 0.689 in the MIMIC-IV cohort. Accuracy was 0.612 (95% confidence interval [CI], 0.595-0.628) versus 0.606 (95% CI 0.600-0.611), specificity 0.574 versus 0.523, sensitivity 0.723 versus 0.754, NPV 0.860 versus 0.842, and PPV 0.364 versus 0.401, respectively. For secondary outcomes, model discrimination was comparable between cohorts. The AUC for in-hospital mortality was 0.727 in the Mayo Cohort versus 0.733 in MIMIC-IV; for IMV, 0.736 versus 0.724; and for NIMV, 0.732 versus 0.708. CONCLUSIONS: Our machine learning algorithm demonstrated good discrimination and a high negative predictive value for identifying low-risk hospitalized CAP patients for respiratory deterioration or death in the external MIMIC-IV dataset, supporting its potential utility for prognostic enrichment in pneumonia clinical trials.