Toward Emotion-Preserving Speech Semantic Communication in Affective Social Systems

With the rapid development of artificial intelligence (AI) and ubiquitous connectivity, speech communication is becoming increasingly important in achieving intelligent interactions between humans, machines, and objects in computational social systems. However, current neural network-based semantic communication frameworks primarily focus on transmitting semantic information while largely overlooking the emotion features in speech communication, which is vital for naturalness and effective interaction in computational social systems. In this article, we propose an emotion-enhanced speech semantic communication system, which effectively enhances the expressiveness and robustness of emotion AI for speech communication. First, we propose an emotion fusion encoding module at the transmitter, where features are dynamically fused via attention mechanisms and subsequently encoded through a channel encoder. Then, we introduce an emotion orthogonal decoding module at the receiver, which reconstructs the fused features via a channel decoder followed by an orthogonally constrained disentanglement network. In addition, a conditional diffusion model guided by emotion features reconstructs high-fidelity speech with enriched emotional expressiveness. Finally, experimental evaluations demonstrate that the proposed framework significantly improves both the bit error rate and the mean opinion score over state-of-the-art models. Furthermore, the system achieves notable reductions in transmission dimensionality.