Structure-driven induction of decision tree classifiers through neural learning

Abstract

The decision tree classifiers represent a nonparametric classification methodology that is equally popular in pattern recognition and machine learning. Such classifiers are also popular in neural networks under the label of neural trees. This paper presents a new approach for designing these classifiers. Instead of following the common top-down approach to generate a decision tree, a structure-driven approach for induction of decision trees, SDIDT, is proposed. In this approach, a tree structure of fixed size with empty internal nodes, i.e. nodes without any splitting function, and labeled terminal nodes is first assumed. Using a collection of training vectors of known classification, a neural learning scheme combining backpropagation and soft competitive learning is then used to simultaneously determine the splits for each decision tree node. The advantage of the SDIDT approach is that it generates compact trees that have multifeature splits at each internal node which are determined on global rather than local basis; consequently it produces decision trees yielding better classification and interpretation of the underlying relationships in the data. Several well-known examples of data sets of different complexities and characteristics are used to demonstrate the strengths of the SDIDT method.