How Reliable Is Sentiment Analysis? A Multi-domain Empirical Investigation

Abstract

Sentiment analysis (also known as opinion mining) is frequently used in monitoring public opinions on the internet. For example, it can help marketers evaluate the success of an ad campaign. It can also be used to assess public opinions during a political campaign. As a result, many businesses and organizations are exploring the potential value of employing sentiment analysis as a part of their business and social intelligence strategies. However, the technology isn’t fully mature yet. As a result, if not used carefully, the results from sentiment analysis can be misleading. In this paper, we present an empirical investigation of the effectiveness of using current sentiment analysis tools to assess people’s opinions in five different domains. The results were very uneven, from decent (e.g., hotel reviews) to poor (e.g., comments on public policies). We also proposed several effectiveness indicators that can be used to signal the appropriateness of using these tools in specific domains.