A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition

Abstract

As an important development direction of natural language processing, emotion recognition in conversation (ERC) remains a challenge in sentiment analysis. Given the large-scale dialogue datasets and their wide application in the fields of recommendation systems and human–machine dialogue systems, researchers have begun to pay more attention to the issue of ERC. In recent research, the task of ERC has been largely based on the graph structure to model the speaker level. However, most existing studies simply splice multimodal features, and the heterogeneity of multimodal features tends to be overlooked. Hence, this paper proposes a multivariate messaging framework to embed heterogeneous information into multimodal relational graphs. In the process of aggregating graph node information, we take into account the homogeneity of nodes and assign different weights to different nodes so as to better aggregate semantic information. In order to improve the robustness of the model, we utilize the mechanism of sharing weights among neighbors to reduce the number of network parameters and improve the generalization ability of the model. In so doing, the node information is aggregated through the constructed graph network, and the final semantic vector representation is obtained. Experiments over two benchmark datasets for ERC show that our proposed model achieves improved performance in accuracy and F1 value.