CitEnergy : A BERT based model to analyse Citizens’ Energy-Tweets

Abstract

Micro-blogging social site Twitter has emerged as a rich source of unstructured text information which could be processed and analysed to extract people’s opinions about several topics and events including natural hazards, energy, sports, transportation, elections etc. The present research study adds a novel perspective in this dimension by extracting citizens’ opinions on several electricity-related issues. Recent research studies in this domain have employed Bag-of-Words (BoW) model for the numerical representation of tweets. However, the BoW model suffers from several issues such as incapable of handling the semantic relationship between words, sparse and high dimensional representation. The present research work overcomes all aforementioned shortcomings by integrating popular word-embeddings with deep learning models for the classification tasks. The study harnesses social media data for two classification tasks: Sentiment classification task and Complaints classification task. Firstly, a series of preprocessing steps are applied on tweets extracted from Twitter streaming API. Subsequently, different word-embeddings models are employed to generate a numerical representation of tweets while capturing the semantic relationship among words. Several deep learning-based sentiment classification models are then deployed on top of generated word-embeddings for identifying/classifying citizens’ sentiments from the tweets. Lastly, the tweets associated with negative sentiment class (identified by the sentiment classification model) are further processed and analysed for building the complaints classification model. The complaints classification model prioritize and assign negative sentiment tweets into one of the two target classes depending on the target issue raised in the tweets (Community level or recurring complaint and Individual level complaint). In addition to this, the current study also proposes Bi-directional Encoded Representations for Transformers (BERT) based Sentiment classification and Complaints classification models for achieving the improved classification accuracy. Experimental evaluation of the proposed BERT based models is done by comparing prediction results with several benchmark deep learning models.