Pruning trees in C-fuzzy random forest

Abstract

Pruning decision trees is the way to decrease their size in order to reduce classification time and improve (or at least maintain) classification accuracy. In this paper, the idea of applying different pruning methods to C-fuzzy decision trees and Cluster–context fuzzy decision trees in C-fuzzy random forest is presented. C-fuzzy random forest is a classifier which we created and we are improving. This solution is based on fuzzy random forest and uses C-fuzzy decision trees or Cluster–context fuzzy decision trees—depending on the variant. Five pruning methods were adjusted to mentioned kind of trees and examined: Reduced Error Pruning (REP), Pessimistic Error Pruning (PEP), Minimum Error Pruning (MEP), Critical Value Pruning (CVP) and Cost-Complexity Pruning. C-fuzzy random forests with unpruned trees and trees constructed using each of these pruning methods were created. The evaluation of created forests was performed on eleven discrete decision class datasets (forest with C-fuzzy decision trees) and two continuous decision class datasets (forest with Cluster–context fuzzy decision trees). The experiments on eleven different discrete decision class datasets and two continuous decision class datasets were performed to evaluate five implemented pruning methods. Our experiments show that pruning trees in C-fuzzy random forest in general reduce computation time and improve classification accuracy. Generalizing, the best classification accuracy improvement was achieved using CVP for discrete decision class problems and REP for continuous decision class datasets, but for each dataset different pruning methods work well. The method which pruned trees the most was PEP and the fastest one was MEP. However, there is no pruning method which fits the best for all datasets—the pruning method should be chosen individually according to the given problem. There are also situations where it is better to remain trees unpruned.