Comparison of four gradient-learning algorithms for neural network Wiener models

Abstract

A gradient-based approach to training neural network Wiener models is presented. Calculation of the gradient or approximate gradient for the series-parallel and parallel Wiener models by the backpropagation, the sensitivity method (SM) and the backpropagation through time (BPTT) is considered in a unified framework. Four different recursive learning algorithms are derived, analysed and compared. For the truncated BPTT, it is shown that the determination of the number of unfolding time steps can be made on the basis of an impulse response function of sensitivity models. Analysis of the computational complexity of these algorithms shows that, contrary to the other recurrent neural network models, computation of the gradient in parallel Wiener models with the sensitivity method or backpropagation through time requires only a little more computational burden than the backpropagation. A simulated data example and a real data example of a laboratory two-tank system are also included to make comparison of different methods and their effectiveness and practical feasibility are shown.