A heterogeneous stacking ensemble based sentiment analysis framework using multiple word embeddings

Abstract

AbstractWord embedding techniques have been proposed in the literature to analyze and determine the sentiments expressed in various textual documents such as social media posts, online product reviews, and so forth. However, it is difficult to capture the entire gamut of intricate inter‐dependencies among words in the textual documents using a specific word embedding technique. In this article, we aim to address this issue by proposing a computation‐efficient stacking ensemble based sentiment analysis framework using multiple word embeddings. The proposed framework uses a combination of three distinct word embeddings generated by three different state of the art word embedding techniques, namely, Word2Vec, GloVe, and BERT for performing the sentiment analysis task. It uses an explicitly trained Word2Vec model to generate the first set of 200‐dimensional word embedding. Similarly, pre‐trained GloVe and BERT models are used to generate the other two sets of 200‐dimensional and a 768‐dimensional word embeddings, respectively. These three distinct word embedding sets are then used to train a heterogeneous stacking ensemble based classifier model comprising LSTM, GRU, and Bi‐GRU based base‐level classifiers, and a LSTM based meta‐level classifier. Experimental results on four different datasets, namely, Sentiment140, IMDB Review, Twitter conversation thread, and Twitter Emotion show that the proposed framework achieves high performance with low false positive rate. The proposed framework is also shown to outperform other sentiment analysis frameworks proposed in the literature.