Credit scoring based on a Bagging-cascading boosted decision tree

Abstract

Establishing precise credit scoring models to predict the potential default probability is vital for credit risk management. Machine learning models, especially ensemble learning approaches, have shown substantial progress in the performance improvement of credit scoring. The Bagging ensemble approach improves the credit scoring performance by optimizing the prediction variance while boosting ensemble algorithms reduce the prediction error by controlling the prediction bias. In this study, we propose a hybrid ensemble method that combines the advantages of the Bagging ensemble strategy and boosting ensemble optimization pattern, which can well balance the tradeoff of variance-bias optimization. The proposed method considers XGBoost as a base learner, which ensures the low-bias prediction. Moreover, the Bagging strategy is introduced to train the base learner to prevent over-fitting in the proposed method. Besides, the Bagging-boosting ensemble algorithm is further assembled in a cascading way, making the proposed new hybrid ensemble algorithm a good solution to balance the tradeoff of variance bias for credit scoring. Experimental results on the Australian, German, Japanese, and Taiwan datasets show the proposed Bagging-cascading boosted decision tree provides a more accurate credit scoring result.