Hybrid CNN-Transformer for Knee Osteoarthritis Severity Grading

The problem of accurately diagnosing of the degree of severity of knee osteoarthritis (KOA) based on simple radiographic images lies in the difficulty of distinguishing between adjacent degrees and differences in imaging conditions, nad in the ordinal nature of the Kellgren-Lawrence (KL) scoring system. In this paper, we use plain radiography (X-ray anteroposterior knee radiographs) as the main type of imaging for KOA analysis. To solve these problems, we introduce an innovative hybrid architecture based on CNNs and transformers with adaptive feature integration and ordinal-aware dual-head learning for KOA degree of severity diagnosis. The novel architecture incorporates a CBAM-ResNeXt-50 model as the backbone network for texture extraction, along with a lightweight transformer-based encoder for modeling the whole anatomy structure. We effectively integrate local and global semantics by designing a learnable adaptive feature fusion module at the image level, producing stage-aware attention on different KOA degrees. Furthermore, we develop an ordinal-aware dual-head learning paradigm that can jointly conduct KL grade classification and continuous KOA severity regression tasks. Experimental results achieve 96.84% accuracy, 0.96 macro F1-score, 0.21 MAE, and 0.959 macro AUC-ROC with fewer adjacent-grade confusions.