User satisfaction with Arabic COVID-19 apps: Sentiment analysis of users’ reviews using machine learning techniques

Abstract

Digital technologies such as mobile health (mHealth) apps with a variety of features can be essential tools for controlling pandemics. Therefore, many Arab countries have launched COVID-19 mHealth apps to reduce the spread of infection among their citizens. Recently, empirical studies have shown that user reviews include useful details to develop apps. However, Arab citizens' satisfaction with the COVID-19 mHealth apps has not been examined yet. Our study aims to provide Arabic sentiment analysis of users’ reviews to explore their satisfaction with Arabic Covid-19 apps. To achieve this goal, we have provided a benchmark dataset composed of 114,499 reviews from 18 Arabic COVID-19 Apps. Six machine learning (ML) models were tested and compared (Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Naive Bayes (NB), Logistic Regression (LR), Random Forest (RF), and Artificial Neural Network (ANN)) using a representative sample of 8220 reviews, which were annotated manually. Then, the best-performing algorithms were applied to the benchmark dataset to explore the polarity of Arab sentiment toward the apps. In a later step, we conducted a thematic analysis of both positive and negative reviews to determine which factors positively and negatively influence the effectiveness of apps. The findings show that the ANN algorithm provides the best performance with 89 % accuracy and 89 % F1. 71 % of user reviews include positive sentiments, while only 21 % include negative sentiments. Frequently crashes, update issues, and bugs were among the most prominent negative factors that affected the effectiveness of apps from the users' point of view. Finally, we presented a set of recommendations to address the negative factors and improve the effectiveness of Arabic COVID-19 apps.