American Politicians Diverge Systematically, Indian Politicians do so Chaotically: Text Embeddings as a Window into Party Polarization

Abstract

Conversations on polarization are increasingly central to discussions of politics and society, but the schisms between parties and states can be hard to identify systematically in what politicians say. In this paper, we demonstrate the use of representation learning as a window into political dialogue on social media through the tweets authored by politicians on Twitter. We propose to embed politicians in a space such that their output embedding vectors represent the content similarity between the two politicians based on their tweets. We further propose a short-text based embedding technique to overcome some of the shortcomings of the previous methods. The learnt embeddings for politicians of India and the United States show two trends. First, that in the US case, we find a clear distinction between Democrats and Republicans, which is also reflected in the coalescing of the states that lean towards each party placing likewise in a graphical space. However, in the Indian case, the federal structure, multiparty system, and linguistic differences clearly manifest in the coalescing political discourse in the largely monolingual north and the scattered regional states. Our work shows ways in which machine learning methods can offer a window into thinking about how polarized political discourses manifest in what politicians say online.