EEG-based Multimodal Emotion Recognition: Recent Progress, Challenges, and Future Directions

Emotion recognition is a crucial part of cognitive computing. Traditional emotion recognition systems include audio-visual modality. However, a recent trend in recognizing emotions is to use physiological signals such as the Electroencephalogram (EEG). EEG signals, together with audio-visual and other physiological signals, improve the performance of emotion recognition systems. This article presents a systematic literature review on EEG-based multimodal (multimedia) emotion recognition systems for the last 5 years. Three major research questions are addressed: (1) What kind of learning models are used in EEG-based multimedia emotion recognition? (2) What are the publicly available related datasets? (3) What are the challenges and future directions of this topic? The answers to the research questions are provided in different subsections.