An optimised decision tree induction algorithm for real world data domains

Abstract

A major weakness of step-wise decision tree (DT) induction algorithms such as ID3 (J.R. Quinlan, 1986) and CHAID (G.V. Kass, 1980) is the lack of a globally optimal search strategy. These algorithms perform a heuristic search which selects the best local attribute/values split for each internal node. No account is taken of the impact on subsequent splits. Once a split is selected, these algorithms have no backtracking mechanism to enable them to change an attribute split. A genetic algorithm (GA) performs a nonlinear search for the optimal or near optimal solution in a pre-defined search space. The paper asserts that GAs are an effective alternative to the step-wise search strategy employed by traditional DT induction algorithms. We present a novel GA based DT induction algorithm that has been applied to three well-known data sets. Results indicate that this algorithm has produced more accurate decision trees.