Hierarchical belief rule-based model for imbalanced multi-classification

Abstract

Classification tasks are of great importance in machine learning. However, class imbalance is a universal problem that needs to be solved in classification and can greatly affect the performance of machine learning classifiers. Developing from the basic belief rule base (BRB) system, the hierarchical belief rule-based system can integrate expert knowledge and has the potential to alleviate the negative effect of class imbalance. To utilize BRB to solve the imbalanced multi-classification task and avoid the combinational explosion problem, a novel hierarchical BRB structure based on the extreme gradient boosting (XGBoost) feature selection method, abbreviated as HFS-BRB is proposed in this paper in order to deal with any number of classes. In the hierarchical BRB structure, there is one main-BRB in the first level and several sub-BRBs in the second level. The XGBoost technique is used for feature selection in the modelling process of each abovementioned BRB model. The output of the main BRB represents the approximated classification between confusable classes. Then, these samples were transmitted to a certain sub-BRB for binary classification to make a precise prediction. Thus, a multi-classification problem can be transformed into several binary classification problems. The class imbalance is alleviated. To test the effectiveness of the proposed method, seven classical benchmark problems for imbalanced classification and a real asteroid orbit classification were performed.