Transformer-based multimodal fusion model predicts early hematoma expansion in spontaneous cerebral hemorrhage: A multicenter study

BACKGROUND: To develop a multimodal transformer model that integrates deep learning, radiomics, and clinical factors for the precise prediction of early hematoma expansion (EHE) in patients with spontaneous intracerebral hemorrhage (sICH). METHODS: We included 465 patients from three hospitals, dividing patients from hospitals 1 and 2 into a training set (n = 315) and an internal testing set (n = 80) in an 8:2 ratio, while using hospital 3 as an external testing set (n = 70). We extracted 2.5D images from non-contrast computed tomography (NCCT) images for deep learning model training and extracted radiomics features from the Region of Interest (ROI). Clinical factors, radiomics features, and deep learning features were incorporated into four machine learning models: Support Vector Machine (SVM), Logistic Regression (LR), Random Forests (RF), and AdaBoost. Finally, we integrated the multimodal features into the transformer model. We evaluated the model's performance using the Area under the curve (AUC), compared model AUCs using the DeLong test, and assessed model consistency with calibration curves. RESULTS: The transformer model achieved the best performance with an AUC of 0.894, outperforming the 2.5D deep learning model (AUC = 0.86) and the radiomics model (AUC = 0.813). The DeLong test revealed significant differences between the transformer model and the radiomics and clinical models. Decision Curve Analysis (DCA) indicated that the transformer model provided the greatest clinical benefit, and calibration curves showed that the fusion model achieved the best predictive value. CONCLUSION: The multimodal transformer model demonstrated excellent performance in predicting EHE, validating the feasibility of multimodal models in predicting EHE.