Learning in dynamic neural networks using signal flow graphs

Abstract

The paper presents the universal approach to the determination of the sensitivity functions for dynamic neural networks and its application in learning algorithms of adaptive networks. The method is based on the application of signal flow graph and specially defined graph adjoint to it. The method is equally applied to either feed-forward or recurrent network structures. This paper is mainly concerned with neural network applications of the approach. Different kinds of dynamic neural networks are considered and discussed in the paper: the FIR dynamic multilayer perceptron (MLP), the cascade connection of dynamic MLPs as well as two non-linear recurrent systems: the dynamic recurrent MLP network and ARMA recurrent network. The rule of sensitivity determination has been applied in practical learning of neural networks. Chosen results of numerical experiments concerning the application of this approach to the learning processes of recurrent neural networks are also given and discussed.