Combination of multiple neural networks using data fusion techniques for enhanced nonlinear process modelling

Abstract

Combining multiple neural networks appears to be a very promising approach in improving neural network generalisation since it is very difficult, if not impossible, to develop a perfect single neural network. This paper proposes using data fusion techniques to combine multiple neural networks and the combination weights change with the model input data. At each given model input data point, the probability of a particular network being the true model is estimated and the network with the highest probability is assigned a combination weight of 1 while other networks assigned combination weights of 0. This probability is calculated using the estimated sum of squared prediction errors of the individual networks on a sliding window covering the most recent sampling times. A nearest neighbour method is used to estimate the network prediction error for a given model input data point. This is to facilitate long range predictions where future prediction errors are unknown and have to be estimated. The proposed techniques are applied to dynamic nonlinear process modelling and modelling of the real world data for the water discharged in Sg Langat Malaysia. Application results demonstrate that the proposed techniques can significantly improve model generalisation especially in long range predictions.