Transformer Based Unsupervised Learning Approach for Imbalanced Text Sentiment Analysis of E-Commerce Reviews

Abstract

In e-commerce, online reviews play a vital role in understanding customer requirements. Companies can perform analysis of these reviews to make right business decisions. However, manually analyzing the content generated by each user on social media is challenging due to its high volume and frequency. Sentiment analysis (SA) is a Natural Language Processing (NLP) technique which provides an automated solution to this problem. It is a challenging subtask of text classification. Over a long time, researchers have proposed different approaches to solve it. This paper proposes an unsupervised learning approach using transformer architecture to perform SA on women’s clothing e-commerce dataset which is imbalanced in nature. In this study, we have fine-tuned DistilBERT, a cutting-edge transformer based pre-trained model and developed models for two subtasks of SA named as Sentiment Classification (SC) and Product Recommendation (PR). The proposed models have achieved the highest F1 scores (0.79 for SC and 0.85 for PR), AUC scores (0.98 for SC and 0.96 for PR) along with the highest accuracy of 0.96 for SC and 0.91 for PR. We found that the performance of our models is least affected by imbalanced dataset issues. The results show that the proposed models have significantly outperformed the traditional supervised approaches and various other existing state-of-the-art (SOTA) models. This study can contribute to a better understanding of consumer sentiment and consumer psychology in the e-commerce transaction business.