Interpretable multi-modal hierarchical framework to support cytomorphological analysis of hematologic cancers

Cytomorphological analysis of bone marrow aspirate (BMA) and peripheral blood smears (PBS) remains a diagnostic cornerstone in hematology. Machine-assisted analysis could support diagnostic workflows, but remains challenging due to the imbalanced prevalence of hematologic malignancies and cytomorphological diversity shaped by biological and therapeutic factors. Here, we extend cell-level multi-instance learning into a hierarchical multi-modal framework, incorporating BMA and PBS, along with recent chemotherapy exposure (yes/no), to classify hematological malignancies into eight major classes. We digitized almost 50,000 pathology slides at ×100 magnification, including 17,000 paired BMA + PBS, covering all hematological diseases and disease phases. The framework achieved high accuracy (AUROC > 0.90) in distinguishing hematological cancers, including acute leukemia, myeloproliferative neoplasms and multiple myeloma. Multi-modal fusion outperformed single-modality baselines, demonstrating the complementary utility of BMA and PBS in resolving uncertainties. Attention mechanisms provided interpretable insights by pinpointing influential cells and morphological patterns. Collectively, we present an intelligent system to assist cytomorphological assessment, support disease classification and enable morphology-based discrimination between malignant and non-malignant marrow states in follow-up settings.