Hybrid CNN-Transformer Model for Enhanced Real-Time Surveillance of Suspicious Group Activities

Effective security and surveillance rely on accurate and timely recognition of suspicious and complex human group activities. This paper proposes a novel hybrid deep learning framework, ResViViT50, for enhanced recognition of such events in real-world surveillance scenarios. The proposed approach integrates a Reduced ResNet-50 architecture for efficient spatial-temporal feature extraction with a Video Vision Transformer augmented by a motion-aware token selection module that prioritizes salient motion tokens, thereby improving long-range dependency modeling and contextual understanding of group interactions. The Reduced ResNet-50 efficiently processes video frames to learn discriminative features related to individual actions, while the ViViT leverages its attention mechanism to model the spatial and temporal relationships between multiple individuals, enabling accurate recognition of group activities and identification of potentially suspicious collective behaviors. Evaluated on the challenging Real-World Fights 2000 and Real-Life Violence Situations datasets, the proposed model achieves competitive recognition accuracy (up to 92.71% on RLVS and 79% on RWF-2000), while maintaining a low computational complexity of 17.23 GFLOPs and efficient inference times of 5724.11 ms on both datasets. These results underscore the efficacy of the approach in enhancing proactive security applications. This comprehensive evaluation ensures that the proposed approach not only delivers high recognition performance but also meets the practical constraints of real-time automated video surveillance systems. The results highlight the efficacy of the proposed approach in enhancing the capabilities of proactive security applications.