Acceleration of learning in feedforward networks using dynamical systems analysis and matrix perturbation theory

Abstract

For the explanation of the dynamical behavior of learning in feedforward networks, the recent work by the authors (1999) has focused on the derivation of a dynamical system model which is valid in the vicinity of temporary minima caused by redundancy of nodes in the hidden layer. The purpose of this paper is to show how to incorporate information from the dynamical system model into a constrained optimization algorithm that will allow prompt abandonment of temporary minima and therefore facilitate learning in feedforward network. It is shown that such a formalism can be obtained by the application of matrix perturbation theory. Experimental results illustrate the analytical conclusions.