Automatic Image Classification of Wireless Capsule Endoscopy Using Two-Branch Feature Aggregation and Patch Self-Attention.

Wireless capsule endoscopy (WCE) is a non-invasive technique for detecting small intestinal diseases and generates approximately 60,000-80,000 gastrointestinal images per examination. Reviewing such a large number of images is time-consuming for clinicians. However, existing approaches typically focus only on local features while neglecting long-range dependencies. To address this limitation, we propose a dual-branch feature aggregation framework that integrates both local and global features for robust WCE image classification. The architecture comprises a CNN branch for local feature extraction and a vision transformer (ViT) branch for global contextual dependencies. An adaptive weighting mechanism is introduced to dynamically calibrate the contribution of each branch ensuring optimal feature representation. Additionally, a novel patch self-attention (PSA) module is designed to focus on representative salient scene regions within local patches. Comprehensive experiments were conducted on three datasets: Kvasir-Capsule, SEE-AI, and OWCE. The proposed framework achieved classification accuracy of 98.31%, 79.33%, and 94.00%; macro-precision of 98.27%, 80.00%, and 94.08%; and macro-F1 of 98.05%, 79.89%, and 94.09%, respectively. To validate the contribution of the adaptive weighting strategy, we conducted a dedicated experiment, which showed that the weighted fusion consistently improves all metrics over its non-adaptive counterpart. Notably, on the challenging SEE-AI, accuracy increased by 2.48%, micro-recall by 1.29%, and micro-precision by 3.47%. These results demonstrate that the proposed dual-branch framework, equipped with adaptive weighting and PSA, delivers robust and generalizable scene classification across different WCE data domains.