Churn prediction in digital game-based learning using data mining techniques: Logistic regression, decision tree, and random forest

Abstract

Educational Technology (EdTech) is an industry that integrates education and technology advances. Digital game-based learning (DGBL) is one of the narrowed-down categories of EdTech. One of the common issues in the EdTech market is the higher churn rate. However, because the DGBL market is still in the early stage, few studies related to marketing perspectives exist. Besides, the approach in education or online gaming industries can be only partially applicable to DGBL. A popular approach for addressing a higher churn rate is churn prediction. By using a dataset from a Japanese company providing DGBL services, this work proposes an approach for the combination of defining churn and churn prediction for DGBL. This work has three objectives. First, determining churn in DGBL by comparing the recency and the addition of average and two standard deviations of user inactive time. Second, clarifying the churn rate of the Japanese service, which became evident as 56.77% by using the newly created churn definition. Third, developing a churn prediction model by comparing logistic regression (LR), decision tree, and random forest models. Feature selection, dataset split ratio comparison, and hyperparameter tuning were conducted to achieve better predictions. Based on the results, LR scored the highest AUC of 0.9225 and an F1-score of 0.9194. These results are on the higher side comparing with the past churn prediction studies in online gaming and education industries. As a consequence, the results indicate the effectiveness of the proposed approach for churn determination and prediction in DGBL.