Fine-Grained Sentiment Classification Using Generative Pretrained Transformer

Abstract

Social media platforms have seen a significant increase in the number of users and content in recent years. Owing to the increased usage of these platforms, incidents of teasing, provocation—both positive and negative—and harassment, and community attacks have increased tremendously. There is an urgent need to automatically identify such content or tweets that can hamper the well-being of an individual or society. Analyzing social media messages from Twitter and Facebook has become the focus of sentiment analysis in recent years, which formerly focused on online product evaluations. Sentiment analysis is used in a wide range of fields besides product reviews, including harassment, stock markets, elections, disasters, and software engineering. After the tweets have been preprocessed, the extracted features are categorized using classifiers like decision trees, logistic regression, multinomial nave Bayes, support vector machines, random forests, and Bernoulli nave Bayes, as well as deep learning techniques like recurrent neural network (RNN) models, long short-term memory (LSTM) models, bidirectional long short-term memory (BiLSTM) models, and convolutional neural network (CNN) model for sentiment analysis. In this paper, different techniques are compared to classify Twitter tweets into three categories: “positive,” “negative,” and “neutral.” We proposed a novel data-balancing technique for text classification. A text classification technique is proposed for analyzing textual data using the Generative Pretrained Transformer model owing to its contextual understanding and more realistic data generation capability. Comparative analysis of different Machine learning and Deep learning models are performed with and without data balancing. The experiments show that the accuracy and F1-measure of the Twitter sentiment classification classifier are improved. The proposed ensemble has outperformed and achieved an accuracy of 90%, precision of 88%, and 81% F1 score.