Behaviour-Aware Hybrid Deep Networks for Detecting Zero-Day and Ransomware Threats

Purpose The rapid escalation of ransomware and zero-day malware attacks poses a significant challenge to conventional signature based detection systems, which cannot generalise to previously unseen threats. This study aims to develop a robust, scalable, and behaviour-aware malware detection framework capable of accurately identifying ransomware and zero day attacks across heterogeneous computing environments. Design methodology approach: A novel multi stage hybrid detection pipeline is proposed that integrates advanced feature selection, deep sequential learning, attention mechanisms, and ensemble classification. Initially, irrelevant and redundant features are eliminated using correlation thresholding, Chi square analysis, mutual information, and variance-based ranking. To capture latent behavioral patterns, a hybrid Gated Recurrent Unit Temporal Convolutional Network GRU-TCN architecture is employed to model long and short term temporal dependencies. These representations are further refined using squeeze-and excitation attention-enhanced TCN blocks. Finally, an XG-Fusion framework that combines GRU encoding, dilated residual TCNs, attention-based feature fusion, and focal loss optimisation is introduced to address class imbalance, with XGBoost serving as a meta-classifier for final decision making. Findings: Experimental evaluations conducted on multiple benchmark datasets demonstrate that the proposed framework consistently outperforms traditional machine learning and baseline deep learning models. Superior performance is achieved in terms of accuracy, precision, recall, F1 Score, and ROC AUC. The hierarchical and attention driven architecture effectively abstracts malicious behavioral patterns and enhances generalization to previously unseen malware variants. Originality: This work introduces a novel multi-stage hybrid deep learning architecture that synergistically combines sequential behavioural modelling, attention-enhanced feature learning, and ensemble-based classification. The proposed approach offers a forward-looking and reliable solution for proactive detection of ransomware and zero-day malware threats.