Binary multi-layer classifier

Abstract

Binary decision trees (BDTs), where each node of the tree is split into two child nodes, are among the most popular classifiers. An alternative type of classification tree, namely, the multi-layer classifier (MLC), has been proposed to split the parent node into 1 or 2 classified child nodes and an unclassified child node at each layer. In contrast to the nodes in a BDT, only the unclassified node of the MLC can be further split. Though the use of MLC is plausible, it has not been widely applied due to a lack of theoretical investigations and thorough tests with real datasets. In this study, we attempt to lay a solid theoretical foundation for a simple MLC with a binary split, i.e., a split into only two nodes, namely, one classified and the other unclassified. Based on the theories developed, we propose a variance-ratio algorithm to construct tree models. The proposed algorithm is thoroughly tested with 40 datasets from well-known repositories. The results indicate that binary MLC models are easier to interpret than other models, achieve significantly better average classification performance than seven other BDT methods and construct fewer tree nodes than most other methods except CTree and NBTree.