Deep Tensor Evidence Fusion Network for Sentiment Classification

Abstract

Recently, a multimodal sentiment analysis of social media has attracted increasing attention, and its core idea is to discovery heuristic fusion strategy to analyze the sentiment orientations over heterogeneous multimodal source from a learned compact multimodal representation. The existing multimodal fusion techniques not only struggle to achieve full heterogeneous data interaction, but also they are unable to dynamically assess the quality of various modal data to determine predictability. In this article, we present a novel deep tensor evidence fusion (DTEF) network for multimodal sentiment classification. First, we propose a common view evaluation network that uses a long short-term memory (LSTM) network and a tensor-based neural network to extract rich intermodal and intramodal information. Then, we propose a unique time cue evaluation network that takes advantage of the temporal granularity associated with numerous pattern sequences. To make reliable decisions, we finally incorporate uncertainty through the trusted fusion layer, which improves the accuracy and robustness of sentimental classification. Our model is validated using the CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) and CMU Multimodal Corpus of Sentiment Intensity (CMU-MOSI) datasets, and the experimental findings demonstrate the superior performance of the proposed network in terms of accuracy compared with the state-of-the-art methods.