UDC-SNN: An Uncertainty-Aware Dynamic Cascading Framework with Spiking Neural Network for Balancing Performance and Energy in Multimodal Emotion Recognition

The aim of this study is to propose an uncertainty-aware dynamic cascading framework based on spiking neural network (UDC-SNN) for multimodal emotion recognition, particularly to address the inherent trade-off between recognition performance and energy efficiency. An asymmetric dynamic routing mechanism was proposed to enable demand-driven activation of the high-power electroencephalogram (EEG) branch, coupled with preliminary inference on a low-power electrocardiogram (ECG) branch and uncertainty quantification via Shannon entropy. Meanwhile, a parameter-free log-linear aggregation strategy was developed to transform modality-specific entropy into dynamic Bayesian weights through an exponential decay function, effectively mitigating the negative transfer effects induced by unimodal noise. The UDC-SNN was evaluated on the multimodal affective dataset DREAMER, comprising 23 subjects (170,660 segments). The averaged recognition accuracy and energy consumption across the three dimensions of valence, arousal, and dominance were 90.75% and 4.62 μJ, respectively. The obtained results suggest that the proposed framework could potentially achieve a favorable balance between high emotion recognition and low energy consumption, thereby establishing its applicability for real-time monitoring in resource-constrained scenarios.