Evolved neural network ensemble by multiple heterogeneous swarm intelligence

Abstract

The neural network ensemble (NNE) is a very effective way to obtain a good prediction performance by combining the outputs of several independently trained neural networks. Swarm intelligence is applied here to model the population of interacting agents or swarms that are able to self-organize. In this paper, we combine NNE and multi-population swarm intelligence to construct our improved neural network ensemble (INNE). First, each component forward neural network (FNN) is optimized by chaotic particle swarm optimization (CPSO) and gradient gescending (GD) algorithm. Second, in contrast to most existing NNE training algorithm, we adopt multiple obviously different populations to construct swarm intelligence. As an example, one population is trained by particle swarm optimization (PSO) and the others are trained by differential evolution (DE) or artificial bee colony algorithm (ABC). The ensemble weights are trained by multi-population co-evolution PSO-ABC-DE chaotic searching algorithm (M-PSO-ABC-DE-CS). Our experiments demonstrate that the proposed novel INNE algorithm is superior to existing popular NNE in function prediction.