Mixed Word Representation and Minimal Bi-GRU Model for Sentiment Analysis

Abstract

In the mission of natural language processing, sentiment analysis is a formidable challenge due to the complexity of deep network architecture and the lack of standard sentiment word representation. In this paper, we proposed a new learning method of the word representation for the comprehensive information of texts and a minimal Bi-GRU (bidirectional gate recurrent unit) model for the task of sentiment classification. First, for capturing sentiment information of words, the supervised three-layer network is used for construct sentiment word representation. We propose the mixed word representation to denote the classification characteristics, which combines the word embedding of neural probabilistic language model with the proposed the sentiment word representation. Next, we propose bidirectional GRU network including forward and backward propagation to consider the semantic relations before and after sentences, meanwhile, to simple the architecture, we apply minimal GRU network. Then, we combine minimal Bi-GRU model with the mixed word representation taking a full account of semantic and sentiment information to classify the sentiment data set as Movie Reviews and IMDB data set. Experimental results demonstrate that the simplicity of the model and superiority of the performance.