MSC-YOLO: An Accurate and Effective Maritime Ship Detection Model Based on Improved YOLOv11n

To address critical challenges in maritime ship detection within complex surveillance imagery, including severe background interference, extreme scale variation, and fine-grained category confusion, this study proposes Maritime Scene Collaborative You Only Look Once (MSC-YOLO), an improved detection model for fixed-location maritime surveillance scenarios. First, a Maritime Scene Adaptive Attention Module (MSAM) is introduced to suppress water-surface clutter and enhance structurally informative ship responses through bidirectional feature regulation, thereby strengthening feature representation in background-complex scenes. In addition, a Scale-aware Dynamic Head (SDA-Head) is designed by integrating deformable convolution with parallel scale-aware prediction branches to improve detection coverage for vessels under pronounced scale variation. Furthermore, a Class Prototype Guided (CPG) module is developed, incorporating class-level prototypes and category-similarity priors to improve the discriminative representation of visually similar ship categories and component states. Experimental results on the constructed maritime surveillance dataset show that MSC-YOLO achieves 0.9723 mAP@50, 0.7315 mAP@50–95, 0.8903 Precision, and 0.9883 Recall. Compared with YOLOv11n, the proposed model improves mAP@50 by 17.77%, Precision by 21.82%, and Recall by 8.16%, indicating clear advantages in target discovery, clutter robustness, and difficult-target coverage in complex maritime surveillance scenes. Visualization and confusion-matrix analyses further show reduced background interference and stronger class-wise discrimination. Overall, MSC-YOLO demonstrates effective and reliable performance for complex maritime surveillance scenarios.