Optimal filtering algorithms for fast learning in feedforward neural networks

Abstract

The training of feedforward networks using the conventional back propagation algorithm is plagued by slow convergence and misadjustment. In this paper, we apply optimal filtering techniques to train feedforward networks in the standard supervised learning framework. We consider first the global problem of computing the synaptic weights all simultaneously, and then develop the idea of local linearization and partitioning. We present three algorithms which are computationally more expensive than standard back propagation, but local at the neuron level. These algorithms do not incorporate any tunable parameters and show excellent performance in comparison to the expensive approach of the global Extended Kalman Algorithm in terms of speed of convergence and quality of solution obtained on three benchmark problems.