Sentiment analysis using TF–IDF weighting of UK MPs’ tweets on Brexit

Abstract

The past decade saw remarkable growth in the production of user-generated text data due to ever-increasing usage of social media. During the same time Twitter has become an indispensable communication tool for politicians. To explore this link, we examine what usage patterns reveal about users’ opinions on the issue of Brexit, these usage patterns consisting of tweet frequency and length, as well as the terms used and their length. We analyse 185,970 tweets from 576 twitter accounts, each account associated with a Member of the British Parliament (MP). We use regression analysis and sentiment analysis, namely Term Frequency–Inverse Document Frequency (TF–IDF), to investigate if there is a relationship between the features of text data and the characteristics of Twitter users. Whereas these methods have previously been applied to American two-party politics, the multiple parties of the British political landscape have led to previous studies using typological analysis (human classifiers) to identify tweets. We present a methodology that assigns a political value based on an MP’s voting record on a single issue (Brexit). We identify systematic yet subtle differences in the way the two sides of the debate use language, but also specific usage patterns that are common to both.