Effect of Synthetic Minority Oversampling Technique (SMOTE), Feature Representation, and Classification Algorithm on Imbalanced Sentiment Analysis

Abstract

The comments received on Internet-based online hotel reservation services are an important resource that can be utilised by hotel service providers including hotel managers' and hoteliers' for exercising quality control measures in their hotel reservation service. Importantly this contributes towards increased customer satisfaction and hotel revisits. In this study, Sentiment Analysis (SA) is used to analyse the comments received from customers. However, there are several problems associated with SA such as the unequal number of each class of data (imbalanced datasets), the classification algorithm and the feature representation. Using SMOTE (Synthetic Minority Oversampling Technique) this research aims to investigate how this technique balances the amount of data from each class employing; the Naive Bayes (NB), Logistic Regression (LR), and Support Vector Machine (SVM) classification algorithms. And also using; term presence (TO), term occurrence (TO), and Term Frequency-Inverse Document Frequency (TF-IDF) feature representations to gauge the effect on the performance of sentiment analysis. The findings from the study found that the use of SMOTE was effective in improving the model's classification performance when data is imbalanced, as evidenced by the average model performance improvement of approximately 12 %. Furthermore, feature representation of TO resulted in an average of 81.68 % of the G-mean Score, followed by TP of 79.89 %, and TF-IDF 79.31 %. As for the classification algorithm, LR resulted in an average score of 81.65 % of the g-mean score, followed by SVM 81.55 %, and NB of 77.68 %.