Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU With Attention Mechanism

Yan Cheng,Guoxiong Xiang,Leibo Yao,Tianwei Tang,Guanghe Zhang,Linhui Zhong

doi:10.1109/access.2020.3005823

Yan Cheng, Guoxiong Xiang + Show 4 more

Open Access

https://doi.org/10.1109/access.2020.3005823

Copy DOI

Abstract

Convolutional Neural Network(CNN) and Recurrent Neural Network(RNN) have been widely used in the field of text sentiment analysis and have achieved good results. However, there is an anteroposterior dependency between texts, although CNN can extract local information between consecutive words of a sentence, it ignores the contextual semantic information between words. Bidirectional GRU can make up for the shortcomings that CNN can't extract contextual semantic information of long text, but it can't extract the local features of the text as well as CNN. Therefore, we propose a multi-channel model that combines the CNN and the bidirectional gated recurrent unit network with attention mechanism (MC-AttCNN-AttBiGRU). The model can pay attention to the words that are important to the sentiment polarity classification in the sentence through the attention mechanism and combine the advantages of CNN to extract local features of text and bidirectional GRU to extract contextual semantic information of long text, which improves the text feature extraction ability of the model. The experimental results on the IMDB dataset and Yelp 2015 dataset show that the proposed model can extract more rich text features than other baseline models, and can achieve better results than other baseline models.

Highlights

Sentiment analysis, known as opinion mining, refers to people’s emotions, opinions, evaluations, attitudes and emotions about services, products, organizations, individuals, problems, events, topics and their attributes [1]
A new multi-channel convolutional neural network and bidirectional GRU model based on attention mechanism are proposed for text sentiment classification; 2)
Yuan et al [41] proposed a sentiment analysis model based on multi-channel convolution and bidirectional GRU networks, and introduced an attention mechanism on the bidirectional GRU to automatically pay attention to features that have a strong influence on sentiment polarity

Summary

INTRODUCTION

Known as opinion mining, refers to people’s emotions, opinions, evaluations, attitudes and emotions about services, products, organizations, individuals, problems, events, topics and their attributes [1]. Cheng et al.: Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU lexicon, and achieve the text sentiment classification This method can achieve text sentiment classification, it is not efficient, because it requires manual construction of an sentiment lexicon and manual labeling. Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) are two popular models in the field of deep learning They have certain defects, the CNN model can’t consider the contextual semantic information, neglecting the interrelationship between the words within the sentence; the RNN model lacks the ability to extract features from the text, and can’t extract local feature well. A new multi-channel convolutional neural network and bidirectional GRU model based on attention mechanism are proposed for text sentiment classification; 2). Introduce attention mechanism on CNN and bidirectional GRU model to allow model automatically extract keyword information in the text, ignoring words that are not relevant to the text classification

RELATED WORK

GRU AND BIDIRECTIONAL GRU

EXPERIMENTAL DATASET

CONCLUSION AND FUTURE WORK