A Testset-Based Method to Analyse the Negation-Detection Performance of Lexicon-Based Sentiment Analysis Tools

Abstract

The correct detection of negations is essential to the performance of sentiment analysis tools. The evaluation of such tools is currently conducted through the use of corpora as an opportunistic approach. In this paper, we advocate using a different evaluation approach based on a set of intentionally built sentences that include negations, which aim to highlight those tools’ vulnerabilities. To demonstrate the effectiveness of this approach, we propose a basic testset of such sentences. We employ that testset to evaluate six popular sentiment analysis tools (with eight lexicons) available as packages in the R language distribution. By adopting a supervised classification approach, we show that the performance of most of these tools is largely unsatisfactory.