Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment Classification

Abstract

A massive volume of unstructured data in the form of comments, opinions, and other sorts of data is generated in real-time with the growth of web 2.0. Due to the unstructured nature of the data, building an accurate predictive model for sentiment analysis remains a challenging task. While various DNN architectures have been applied to sentiment analysis with encouraging results, they suffer from high dimensional feature space and consider various features equally. State-of-the-art methods cannot properly leverage semantic and sentiment knowledge to extract meaningful relevant contextual sentiment features.This paper proposes a sentiment and context-aware hybrid DNN model with an attention mechanism that {intelligently learns and highlights salient features of relevant sentiment context in the text. We first use integrated wide coverage sentiment lexicons to identify text sentiment features then leverage bidirectional encoder representation from transformers to produce sentiment-enhanced word embeddings for text semantic extraction. After that, the proposed approach adapts the BiLSTM to capture both word order/contextual text semantic information and the long-dependency relation in the word sequence. Our model also employs an attention mechanism to assign weight to features and give greater significance to salient features in the word sequence. Finally, CNN is utilized to reduce the dimensionality of feature space and extract the local key features for sentiment analysis. The effectiveness of the proposed model is evaluated on real-world benchmark datasets demonstrating that the proposed model significantly improves the accuracy of existing text sentiment classification.