Word Embedding Dropout and Variable-Length Convolution Window in Convolutional Neural Network for Sentiment Classification

Abstract

Recently the research on sentiment analysis has been attracting growing attention because of the popularity of opinion-rich resources, such as internet movie databases and e-commerce websites. Convolutional neural network(CNN) has been widely used in sentiment analysis to classify the polarity of reviews. For deep convolutional neural networks, dropout is known to work well in the fully-connected layer. In this paper, we use dropout technique in the word embedding layer, and proof it is equivalent to randomly picking activation based on a multinomial distribution at training time. Empirical results also support this and show that using dropout in the word embedding layer can reduce over-fitting. Meanwhile, we investigate the effect of convolution window size on the classification results, and use variable-length convolution window in proposed method. Experimental results show that our method obtains a state-of-the-art performance on ASR. Compared with other similar architectures, the accuracies of our method in this paper are also competitive on IMDB and Subj.