Attentive Intra-modality Fusion for Multimodal Sentiment Analysis

Abstract

AbstractThe growing trend of sharing opinion videos on social media platforms leads to more and more attention to multimodal sentiment analysis research. A number of approaches in multimodal sentiment analysis have been proposed and continual improved results were produced. Most of the existing studies have paid more attention to cross-modalities fusion, ignoring valuable information fusion within each uni-modality. To this end, we propose an AGTL (Attentive Global and Temporal Local information) component to capture the global semantic information and the local semantic representation containing temporal relationships and get the different contributions of global or local information in each uni-modality. After later fusion of representations of different modalities learned by AGTL, we get the final representation of multi modalities. Different comparative and ablation experiments are conducted on a public multimodal sentiment analysis dataset, and the experimental results demonstrate the effectiveness of our model in multimodal sentiment analysis.KeywordsSentiment analysisMultimodal fusionIntra-modality fusion