Bidirectional Complementary Correlation-Based Multimodal Aspect-Level Sentiment Analysis

Abstract

Aspect-based sentiment analysis is the key to natural language processing, and it focuses on the polarity of emotions associated with specific text aspects. Traditional models that combine text and visual data tend to ignore the deeper interconnections between patterns. To solve this problem, the authors propose a multimodal sentiment-oriented analysis (BiCCM-ABSA) model based on bidirectional complementary correlation. The model utilizes text-image synergy through a novel cross-modal attention mechanism to align text with image features. With the transformer architecture, it is not only a simple fusion, but also ensures the complex alignment of multi-modal features and gating mechanisms. Experiments were conducted on the Twitter-15 and Twitter-17 datasets, achieving 69.28 accuracy and 67.54% F1 score, respectively. The experimental results demonstrate the advantages of BiCCM-ABSA, the bidirectional approach of the model and the effective cross-modal correlation set a new benchmark in the field of multimodal emotion recognition, providing insights beyond traditional single-modal analysis.