Research on Predictive Model Based on Ensemble Learning

Abstract

Ensemble learning completes learning tasks by building and combining multiple learners. The use of ensemble learning can make accurate prediction. This paper used the dataset publicly available on kaggle platform. Firstly, this paper preprocessed and performed descriptive statistics on the dataset, based on which this research constructed the prediction model. Three ensemble learning models Random Forest, AdaBoost, and LightGBM were selected to study the data. To prevent overfitting, a 10-fold cross-validation method was used to train the classifiers and the models were tuned using the grid search method. Finally, the three models were compared in terms of Accuracy, Precision, Recall, F1-score, ROC curve and AUC values. The comparison shows that all three models have good performance, and the accuracy of all model predictions are higher than 80%. However, there is a slight difference in classification ability among the models. Random Forest performs best, with an Accuracy of 86.94, Precision of 85.91, Recall of 93.10, F1-score of 0.8936, and AUC of 0.8906. All evaluation indexes are higher, which also verify the feasibility of using ensemble learning algorithms in prediction.