Design Smart NNTrees Based on the R⁴-Rule

Abstract

Neural network tree (NNTree) is a hybrid learning model with the overall structure being a decision tree (DT), and each non-terminal node containing an expert neural network (ENN). Generally speaking, NNTrees outperform conventional DTs because more complex and possibly better features can be extracted by the ENNs. So far we have studied several genetic algorithms (GAs) for designing the NNTrees. These algorithms are computationally expensive, and the NNTrees obtained are often very large. In this paper, we propose a new approach based on the R/sup 4/-rule, which is a non-genetic evolutionary algorithm proposed by the author several years ago. The key point is to propose a heuristic method for defining the teacher signals for the examples assigned to a non-terminal node. Once the teacher signals are defined, the ENNs can be trained quickly using the R/sup 4/-rule. Experiments with several public databases show that the new approach can produce smart NNTrees quickly and effectively.