Multi-model deep transfer learning and hybrid BiLSTM-AM-DCN with Walrus optimization algorithm for multi-class retinal disease classification in OCT images

The manual interpretation of Optical Coherence Tomography (OCT) images by ophthalmologists is a subjective and labor-intensive process, driving the need for accurate automated diagnostic systems. Despite the proliferation of deep learning methods in this domain, classification accuracy and generalization capabilities still require considerable enhancement. In this study, we introduce an automated framework for the detection and classification of retinal diseases using OCT images. This work employs deep transfer learning to enhance four pretrained models: MobileNet-V2, ResNet-50, EfficientNet-B0, and DenseNet-161. For comparison, we also propose a novel hybrid network that integrates a bidirectional LSTM with an attention mechanism within a Deep & Cross Network framework (BiLSTM-AM-DCN). After feature extraction, feature vectors are refined using the Walrus Optimization Algorithm (WaOA) to select the most discriminative characteristics. Evaluated on a clinically curated OCT dataset, we report results from two experimental settings. Setting 1 (five-fold cross-validation): The full BiLSTM-AM-DCN model achieves 98.84% accuracy, compared to 96.88% without WaOA and 91.23% for the CNN baseline. Setting 2 (test performance after WaOA optimization): The BiLSTM-AM-DCN achieves 95.54% accuracy, while EfficientNet-B0 with PCA → SVM achieves 94.14%, MobileNet-V2 achieves 93.57%, DenseNet-161 achieves 93.14%, and ResNet-50 achieves 92.71%. The BiLSTM-AM-DCN model achieves low standard deviations (±0.19–0.63) and cross-dataset validation shows only 2.4–3.4% accuracy drop on unseen OCT datasets, demonstrating strong potential for clinical deployment.