Lexicon Ensemble and Lexicon Pooling for Sentiment Polarity Detection

Abstract

ABSTRACTThis paper presents our experimental work towards detecting sentiment polarity of free-form texts: first by using an ensemble of sentiment lexicons and then through a lexicon pooled machine learning classifier. In the ensemble design, we combined four different sentiment lexicons in different ways to determine sentiment polarities of different text data. The ensemble approach, however, did not achieve superior performance as initially thought. Therefore, in the second design, we tried to pool the sentiment lexicon knowledge into the machine learning classification process itself of a multinomial naive Bayes classifier. The experimental designs are evaluated on three document and two sentence datasets. The lexicon pooled approach obtains superior accuracy levels as compared to standard naive Bayes classifier as well as lexicon-based methods. Furthermore, as the amount of training data decreases, the accuracy levels of lexicon pooled machine learning classifier decays slowly as compared to standalone naive Bayes classifier. The framework presented proves useful and robust and can be extended to any classification task.