Multistage Neural Networks: Adaptive Combination of Ensemble Results

Abstract

In the past decade, more and more research has shown that ensembles of neural networks (some times referred to as committee machines or classifier ensembles) can be superior to single neural network models, in terms of the generalization performance they can achieve on the same data sets. In this paper, we propose a novel trainable neural network ensemble combination schema: multistage neural network ensembles. Two stages of neural network models are constructed. In the first stage, neural networks are used to generate the ensemble candidates. The second stage neural network model approximates a combination function based on the results generated by the ensemble members from the first stage. A sample of the data sets from UCI Machine Learning Depository are modeled using multistage neural networks and a comparison of the performance between multistage neural networks and a majority voting scheme is conducted.