An ensemble method with sentiment features and clustering support

Abstract

Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) are efficiently applied to natural language processing, especially sentiment analysis. CNN employs filters to capture local dependencies while LSTM designs a cell to memorize long-distance information. However, integrating these advantages into one model is challenging because of overfitting in training. To avoid this problem, we propose a freezing technique to learn sentiment-specific vectors from CNN and LSTM. This technique is efficient for integrating the advantages of various deep learning models. We also observe that semantically clustering documents into groups is more beneficial for ensemble methods. According to the experiments, our method achieves competitive results on the five well-known datasets: Pang & Lee movie reviews, Stanford Twitter Sentiment and Stanford Sentiment Treebank for sentence level, IMDB large movie reviews, and SenTube for document level.