A novel Boosted-neural network ensemble for modeling multi-target regression problems

Abstract

In this paper, the concept of ensemble learning is adopted and applied to modeling multi-target regression problems with high-dimensional feature spaces and a small number of instances. A novel neural network ensemble (NNE) model is introduced, called Boosted-NNE based on notions from boosting, subspace projection methods and the negative correlation learning algorithm (NCL). Rather than using an entire feature space for training each component in the Boosted-NNE, a new cluster-based subspace projection method (CLSP) is proposed to automatically construct a low-dimensional input space with focus on the difficult instances in each step of the boosting approach. To enhance diversity in the Boosted-NNE, a new, sequential negative correlation learning algorithm (SNCL) is proposed to train negatively correlated components. Furthermore, the constrained least mean square error (CLMS) algorithm is employed to obtain the optimal weights of components in the combination module. The proposed Boosted-NNE model is compared with other ensemble and single models using four real cases of multi-target regression problems. The experimental results indicate that using the SNCL in combination with the CLSP method offers the capability to improve the diversity and accuracy of the Boosted-NNE. Thus, this model seems a promising alternative for modeling high-dimensional multi-target regression problems.