Functional and Clinical: An Explainable Deep Learning Model for Multimodal Alzheimer's Disease Classification

PURPOSE: Functional magnetic resonance imaging (fMRI) and deep learning models can classify Alzheimer's disease (AD) with high accuracy. These models are highly adaptable and work with a plethora of architectures, data types, and AD stages. However, fMRI deep learning models lack clinical application due to issues with small datasets, explainability, and reliability (e.g., data leakage). METHODS: In this study, we address these issues using multimodal and explainable artificial intelligence (XAI) methods. Specifically, we overcome data size limitations by supplementing fMRI data with clinical tests, use a strict leave-one-out cross-validation regime to control for data leakage, and apply perturbation ranking to explain the importance of features in our model. Our 3D convolutional neural network model was trained and validated on 52 participants from ADNI using five clinical tests and fMRI of the default mode network. FINDINGS: The resulting multimodal model classified AD from controls with an accuracy of 90% and outperformed the same architecture without clinical data (58% accuracy). Our feature rankings showed that clinical tests changed in importance within our model depending on the diagnostic group. For example, our model found the MoCA to be highly important for classifying controls but not for AD. This trend of feature importance was seen across almost all fMRI and clinical features. CONCLUSION: Our model was highly accurate and highlighted the importance of combining fMRI and clinical data for AD classification. These findings have implications for the refinement of multimodal deep learning models; however, our small sample and need for external validation are also noted.