Analysis of sentiment in tweets addressed to a single domain-specific Twitter account: Comparison of model performance and explainability of predictions

Abstract

Many institutions and companies find it valuable to know how people feel about their ventures; hence, scientific research in sentiment analysis has been intensely developed over time. Automated sentiment analysis can be considered as a machine learning (ML) prediction task, with classes representing human affective states. Due to the rapid development of ML and deep learning (DL), improvements in automatic sentiment analysis performance are achieved almost every year. Since 2013, Semantic Evaluation (SemEval) has hosted a worldwide community-acknowledged competition that allows for comparisons of recent innovations. The sentiment analysis tasks focus on assessing sentiment in Twitter posts authored by various publishers and addressing multiple subjects. Our study aimed to compare selected popular and recent natural language processing methods using a new data set of Twitter posts sent to a single Twitter account. For improved comparability of our experiments with SemEval, we adopted their metrics and also deployed our models on data published for SemEval-2017. In addition, we investigated if an unsupervised ML technique applied for the detection of topics in tweets can be leveraged to improve the predictive performance of a selected transformer model. We also demonstrated how a recent explainable artificial intelligence technique can be used in Twitter sentiment analysis to gain a deeper understanding of the models’ predictions. Our results show that the most recent DL language modeling approach provides the highest quality; however, this quality comes at reduced model transparency.