On-line learning algorithms for neural networks with IIR synapses

Abstract

This paper is focused on the learning algorithms for dynamic multilayer perceptron neural networks where each neuron synapsis is modelled by an infinite impulse response (IIR) filter (IIR MLP). In particular, the backpropagation through time (BPTT) algorithm and its less demanding approximated on-line versions are considered. In fact it is known that the BPTT algorithm is not causal and therefore can be implemented only in batch mode, while many real problems require on-line adaptation. In this paper the authors give the complete BPTT formulation for the IIR MLP, derive an already known on-line learning algorithm as a particular approximation of the BPTT, and propose a new approximated algorithm. Several computer simulations of identification of dynamical systems are also presented to assess the performance of the approximated algorithms and to compare the IIR MLP with more traditional dynamic networks.