The Construction of Scalable Decision Tree based on Fast Splitting and J-Max Pre-Pruning on Large Datasets

Abstract

The decision tree is one of the most important algorithms in the classification which offers a comprehensible model of data. In building a tree we may encounter a memory limitation. The present study aimed to implement an incremental scalable approach based on fast splitting and present pruning to construct the decision tree on a large dataset as the complexity of the tree decreases. The proposed algorithm constructs the decision tree without storing the entire dataset in the primary memory by using a minimum number of parameters. Furthermore, the J-max Pre pruning method was used to reduce the complexity with acceptable results. Experimental results show that this approach can create a balance between the accuracy and complexity of the tree and overcome the difficulties of the complexity of the tree. In spite of the appropriate accuracy and time, the proposed algorithm could produce a decision tree with less complexity on the large dataset.