Non-sequential partitioning approaches to decision tree classifier

Abstract

Decision tree is a well-known classifier which is widely used in real-world applications. It is easy to interpret, however it suffers from instability and lower classification performance for high-dimensionality datasets due to curse of dimensionality. Feature set partitioning is a novel concept to address the higher dimensionality problem by dividing the feature set into subsets (blocks). Many of the existing partitioning based decision tree approaches are sequential in nature, which lack logical relationships amongst the features. In this work, we propose novel non-sequential feature set partitioning methods by exploiting the ideas of Ferrer Diagram and Bell Triangle to create feature blocks with a mix of low, medium, and high correlation features. The experimental results on 11 UCI and KEEL datasets demonstrate the superiority of the proposed partitioning methods, upto 5% higher classification accuracy, over NBTree, BFTree, Serial-CMFP partitioning method, and classical decision tree techniques. The proposed methods also exhibit improved stability as compared to other decision tree methods.