A hybrid scheme-based one-vs-all decision trees for multi-class classification tasks

Abstract

Decision tree algorithms have been proved to be a powerful and popular approach in classification tasks. However, they do not have reasonable classification performance in multi-class scenarios. In the present study, decision tree algorithms are combined with the one-vs-all (OVA) binarization technique to improve the generalization capabilities of the scheme. However, unlike previous literature that has focused on aggregation strategies, the present study is focused on the process of building base classifiers over the OVA scheme. A novel split criterion, entitled by the splitting point correction matrix (SPCM), is proposed in this regards, which can effectively deal with the unbalance problem caused by the OVA scheme.The SPCM is a kind of hybrid scheme, which integrates distribution and permutation information from the training data at each splitting point. Therefore, compared to other classical split criteria, such as the C4.5, the proposed method can make the right choice about the optimal splitting point at the root or internal nodes from multi-angle.In order to evaluate the effectiveness of the SPCM approach, extensive experiments are carried out compared to the classical and state-of-the-art methods. The experiments are performed on sixteen datasets, where the effectiveness and the accuracy of the proposed method is verified. It is concluded that the SPCM method not only has excellent classification performance but also produces a more compact decision tree. Moreover, it is found that the SPCM method has especially a considerable improvement in the depth of tree.