A Pearson’s correlation coefficient based decision tree and its parallel implementation

Abstract

In this paper, a Pearson’s correlation coefficient based decision tree (PCC-Tree) is established and its parallel implementation is developed in the framework of Map-Reduce (MR-PCC-Tree). The proposed methods employ Pearson’s correlation coefficient as a new measure of feature quality to confirm the optimal splitting attributes and splitting points in the growth of decision trees. Besides, the proposed MR-PCC-Tree adopts Map-Reduce technology to every component during the decision trees learning process for parallel computing, which mainly consists of a parallel Pearson’s correlation coefficient based splitting rule and a parallel splitting data method. The experimental analysis is conducted on a series of UCI benchmark data sets with different scales. In contrast to several traditional decision tree classifiers including BFT, C4.5, LAD, SC and NBT on 17 data sets, the proposed PCC-Tree is no worse than the traditional models as a whole. Furthermore, the experimental results on other 8 data sets show the feasibility of the proposed MR-PCC-Tree and its good parallel performance on reducing computational time for large-scale data classification problems.