Deep Hybrid Neural Networks with Improved Weighted Word Embeddings for Sentiment Analysis

Abstract

Deep learning models such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) models have recently emerged as effective solutions to various NLP tasks with comparatively remarkable results. The CNN model efficiently extracts higher level features using convolutional layers and max-pooling layers while the LSTM model allows capturing long-term dependencies between word sequences. In this paper, we propose a hybrid CNN-LSTM model taking advantage of both models to overcome the sentiment analysis problem on Twitter data. We employ a multi-channel CNN that extracts local n-gram features in different sizes using several filters of different lengths. We also employ a weighted average word embeddings method which incorporates sentiment information in the continuous representation of words based on an adapted version of the delta TFIDF measure. These word representations will be the input for the CNN-LSTM model. Experiment results substantiate our intuition by reaching good macro average recall and accuracy scores beating several existing models as well as our same model using individual CNN and LSTM.