Multimodal sentiment analysis based on fusion methods: A survey

Abstract

Sentiment analysis is an emerging technology that aims to explore people’s attitudes toward an entity. It can be applied in a variety of different fields and scenarios, such as product review analysis, public opinion analysis, psychological disease analysis, and risk assessment analysis. Traditional sentiment analysis only includes the text modality and extracts sentiment information by inferring the semantic relationship within sentences. However, some special expressions, such as irony and exaggeration, are difficult to detect via text alone. Multimodal sentiment analysis contains rich visual and acoustic information in addition to text, and uses fusion analysis to more accurately infer the implied sentiment polarity (positive, neutral, negative). The main challenge in multimodal sentiment analysis is the integration of cross-modal sentiment information, so we focus on introducing the framework and characteristics of different fusion methods. In addition, this article discusses the development status of multimodal sentiment analysis, popular datasets, feature extraction algorithms, application areas, and existing challenges. It is hoped that our work can help researchers understand the current state of research in the field of multimodal sentiment analysis, and be inspired by the useful insights provided in the article to develop effective models.