SemCap: Sentiment-aware semantic captioning for multimodal aspect-based sentiment analysis

Multimodal aspect-based sentiment analysis (MABSA) is critical for fine-grained opinion mining from social media. Current mainstream methods rely on feature-level fusion and attention mechanisms to align visual and textual features. However, MABSA suffers from visual noise caused by cluttered image fusion and textual noise caused by inter-aspect interference, leading to modality conflicts and spurious aspect predictions. We propose SemCap, a framework that mitigates these issues using a sentiment-aware caption as an explicit, high-level visual proxy. This caption is generated by Qwen3-VL under sentiment-oriented prompts to selectively describe emotion-bearing entities. To accurately aggregate cross-modal sentiment, we design the Semantic Proxy Dynamic Routing (SPDR) fusion module, which uses the caption representation to guide the selection of relevant visual patches. The refined features are input to a multimodal interactive decoder, augmented by auxiliary supervision on aspect boundaries and polarity labels to enhance localization and discrimination. Extensive experiments on the Twitter-2015 and Twitter-2017 datasets demonstrate SemCap’s superior performance over existing MABSA methods.