AMIFN: Aspect-guided multi-view interactions and fusion network for multimodal aspect-based sentiment analysis

Abstract

Aspect-based sentiment analysis (ABSA), which aims to analyze users’ sentiment towards the targeted aspect, has recently gained increasing attention due to its importance in supporting corresponding decision-makings in various tasks. Most existing ABSA studies primarily depend on only textual modality, but ignore the fact that in many cases the targeted aspect doesn’t appear in the sentence. Thus, multimodal ABSA is expected to alleviate this dilemma. However, most existing MABSA approaches still suffer from the following limitations: (1) ignoring the possible aspect-image irrelevant issue; (2) ignoring the coarse-grained interaction between the sentence and its associated image; (3) failing to simultaneously leverage multiple types of useful knowledge information. To address these issues, we propose an aspect-guided multi-view interactions and fusion network (AMIFN) for MABSA. Specifically, we utilize the multi-head attention mechanism to generate aspect-guided textual representation, which is used as the extended aspect semantic for guiding the subsequent aspect-related interactions. When exploring aspect-guided visual representation, we employ the image gate to dynamically filter potential noise introduced by the associated image to generate the final image representation. Meanwhile, the coarse-grained sentence-image interaction, which contains context and semantics information, and the syntactic dependencies, are leveraged for graph construction to obtain aspect-guided text-image interaction representations. Finally, the extracted multi-view interaction representations are integrated for sentiment classification. Extensive experimental results on three multimodal benchmark datasets demonstrate the superiority and rationality of AMIFN.