FT4cip: A new functional tree for classification in class imbalance problems

Abstract

Decision trees (DTs) are popular classifiers partly due to their reasonably good classification performance, their ease of interpretation, and their widespread use in ensembles. To improve the classification performance of individual DTs, researchers have used linear combinations of features in inner nodes (Multivariate decision trees), leaf nodes (Model trees), or both (Functional trees). In this paper, we present a new functional tree, Functional Tree for class imbalance problems (FT4cip). FT4cip is designed to work with class imbalance problems, where one of the classes in the database has few objects compared to another class. FT4cip achieves better classification performance, in terms of AUC, than the best model tree (LMT) and functional tree (Gama) that we identified. The statistical comparison was made in 110 databases using Bayesian statistical tests. We also make a meta-analysis of classification performance per type of database, which helps us recommend a classifier given a problem. We show how each design decision taken when building FT4cip contributes to classification performance or simple models, and rank them according to their importance to classification performance. To avoid a problem of fragmentation in DT literature, we contrast each design decision taken when building FT4cip against LMT and Gama.