Imbalanced aspect categorization using bidirectional encoder representation from transformers

Abstract

Sentiment analysis (also called opinion mining) is one of the widely used research fields of natural language processing. E-commerce service providers use this technique to analyze the sentiment of a product or a service in texts, posts, and comments. In particular, the service providers and users want to understand the sentiment on product aspect categories rather than the overall sentiment of a product. These aspect categories encounter the class imbalance problem. Therefore, the BERT (Bidirectional Encoder Representation from Transformers) based fine-tuning model is presented to deal with the imbalanced aspect categorization task. Specifically, this paper studies various data sampling techniques such as stratified random sampling (SRS), random undersampling (RUS), and random oversampling (ROS) for reducing the class imbalance problem. Empirically, the results show that the proposed BERT fine-tuning model with the SRS technique achieves better results. In particular, the model achieves 96.21% for the validation and 96.47% for testing using the news aggregator data. Similarly, the SMS spam collection data achieves 99.20% for the validation and 99.10% for testing.