Integrating Multimodal Learning and Explainable AI for Enhanced and Interpretable Prostate Lesion Classification

Abstract Artificial Intelligence systems could find many important applications in the medical field, holding excellent potential for improving disease diagnosis, treatment identification and selection. These opportunities are often jeopardized by the lack of interpretability of such systems, slowing down AI adoption. To overcome the issue, we first introduce an analytical framework exploiting multimodal deep learning for the classification of prostate lesions using Magnetic Resonance Imaging (MRI) data and clinical information on the patients. Then, we propose a multimodal explainability approach based on visual explanations to interpret the proposed model decision-making process and identify how the different modalities contribute to each specific prediction. Our findings, based on the PI-CAI Grand Challenge dataset, demonstrate the potential of combining multimodal data with eXplainable AI (XAI) to enhance prostate cancer diagnosis, improving model predictive performance, interpretability and understanding in treatment decision-making.