Chuanzhong Xuan, Junze Jia, Suhui Liu, Zhaohui Tang
Intelligent sheep behavior monitoring is vital for modern husbandry, but faces severe challenges in natural pastures due to high-density flock occlusion. Traditional 2D skeleton-based networks often suffer from depth ambiguity and feature collapse, misclassifying static tremors as dynamic displacement. To overcome this, we propose a robust multimodal dual-stream framework using skeletal and local visual fusion. The architecture features an upstream spatial perception stage utilizing YOLOv11m-Pose. To reduce annotation costs and improve robustness, we introduce an Active Hard-Example Mining mechanism, explicitly retaining difficult samples with severe overlapping or edge truncation. For downstream behavioral decisions, a multimodal dual-stream architecture processes the targets. The Spatio-Temporal Kinematic Stream employs a Kinematic Denoising Engine, incorporating a 1D Gaussian filter and displacement dead-zone gate to purify 2D coordinates before feeding them into a BiLSTM network. Concurrently, the Spatial Visual Stream uses a ResNet-50 backbone on cropped RGB patches to capture essential spatial context, addressing the limitations of pure coordinates. Finally, a weighted Softmax layer integrates both streams. Experiments on a complex real-world dataset validate this approach. A baseline kinematic-only model achieved just 69.05% overall accuracy and 68.18% walking precision. In contrast, our dual-stream fusion network achieved 93.26% overall accuracy, elevating walking precision to 97.14% and the eating F1-score to 94.29%. By effectively decoupling similar static and dynamic behaviors, this study demonstrates the indispensability of local visual features, establishing a high-precision baseline for smart livestock monitoring.