Oscar Fidel Antunez Martinez, Shih-Chun Lin
An explainable Random Forest model using routinely collected nursing assessment variables outperformed Logistic Regression and showed satisfactory calibration. Explainable machine learning may support nursing triage and early clinical decision-making, although external validation is required before clinical implementation.
BACKGROUND: Early identification of children requiring hospital admission is essential for safe pediatric emergency care. Existing prediction models often rely on information unavailable during the initial nursing assessment or lack clinical interpretability.
OBJECTIVES: To develop and internally validate an explainable machine learning model for predicting hospital admission among children presenting to emergency departments using routinely collected nursing assessment variables.
DESIGN: Retrospective secondary analysis of a nationally designed probability cross-sectional survey.
METHODS: Data were obtained from the 2022 National Hospital Ambulatory Medical Care Survey Emergency Department Public Use File. Pediatric visits (0-17 years) were included. Six supervised machine learning algorithms were compared. The best-performing model underwent hyperparameter optimization with five-fold cross-validation and was evaluated in an independent testing cohort. Model explainability was assessed using SHapley Additive exPlanations.
RESULTS: Among 3427 pediatric emergency department visits, 176 (5.1%) resulted in hospital admission. The optimized Random Forest achieved the best performance (ROC-AUC = 0.720, 95% CI 0.648-0.790; PR-AUC = 0.147, 95% CI 0.087-0.220), with 89.9% accuracy, 19.3% precision, 30.2% recall, and an F1-score of 0.235. Calibration was satisfactory (Brier score = 0.047; calibration slope = 0.903). The model significantly outperformed Logistic Regression (P = 0.0005). Pulse rate, respiratory rate, waiting time, body temperature, blood pressure, triage urgency, and emergency medical services arrival were the most influential predictors.
CONCLUSIONS: An explainable Random Forest model using routinely collected nursing assessment variables outperformed Logistic Regression and showed satisfactory calibration. Explainable machine learning may support nursing triage and early clinical decision-making, although external validation is required before clinical implementation.