DANA App Sentiment Analysis: Comparison of XGBoost, SVM, and Extra Trees

Abstract

This research aims to analyze sentiment on DANA application reviews to find out user perceptions by comparing Extreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), and Extra Trees Classifier classification methods. DANA application review data is obtained from the Kaggle site which consists of 50,000 Indonesian-language reviews labeled with positive and negative sentiments. The research stages include data preprocessing to clean and prepare the review text, applying word weighting using Word2Vec to give weight to words based on their context, balancing sentiment classes using SMOTE to address the imbalance of positive and negative review classes. It should be noted that the initial proportion of data before applying SMOTE may affect the results. The data is then divided into training and testing sets, then the models are trained and evaluated using Confusion Matrix and K-Fold Cross-Validation. The results of the three classification methods are measured by the accuracy matrix and F1-Score to assess model performance, the SVM and XGBoost methods obtained an accuracy of 93% and the ETC method achieved an F1-Score value of 96% at K=6, the three models proved to be very accurate in predicting the sentiment of DANA application reviews both positive and negative. The practical implications of this research can identify areas for application improvement, develop popular features, personalize services based on user preferences, and manage application reputation.