Ai-powered virtual reality commerce: temporal aggregation for deep facial expression recognition

Virtual reality (VR) is transforming e-commerce by enabling immersive, interactive product experiences that surpass the limitations of traditional two-dimensional platforms. However, measuring user engagement in these dynamic environments remains a challenge, particularly as current metrics often overlook subtle emotional cues. To address this gap, this paper presents an AI-powered framework that leverages vision-based deep learning and novel temporal aggregation techniques to measure user interest during VR shopping experiences. This study builds on a rigorously validated convolutional neural network (CNN) using the Xception architecture and applies the model to a desktop (i.e., not head-mounted display, HMD) VR study, enabling evaluation of temporal aggregation; HMD occlusion is left for future work. Participants (N=46) explored virtual showrooms featuring Apple and Tesla products while their faces were recorded. To assess the effectiveness of different temporal aggregation strategies, frame-level predictions of user interest were combined into video-level classifications using six methods: no aggregation (baseline), mean, median, Gaussian-weighted, peak-weighted, and trend-aware attention. Results demonstrate that trend-aware attention outperforms all other techniques, achieving 87% accuracy, 95% recall, and F1 score of 0.90. The trend-aware method effectively models both the intensity and temporal dynamics of facial expressions, allowing for time-resolved detection of fluctuating engagement signals across the session. These findings have important implications for VR-based e-commerce, enabling retailers to optimize virtual store layouts, personalize product recommendations, and enhance user interactions based on emotion-aware feedback. More broadly, this scalable, data-driven approach can improve user experience in diverse immersive applications, including education, gaming, training, and healthcare. By integrating AI and temporal modeling, this research advances the development of emotionally intelligent VR systems that can analyze responsive user engagement.