Feedforward Training Research Articles

Optimal filtering algorithms for fast learning in feedforward neural networks

The training of feedforward networks using the conventional back propagation algorithm is plagued by slow convergence and misadjustment. In this paper, we apply optimal filtering techniques to train feedforward networks in the standard supervised learning framework. We consider first the global problem of computing the synaptic weights all simultaneously, and then develop the idea of local linearization and partitioning. We present three algorithms which are computationally more expensive than standard back propagation, but local at the neuron level. These algorithms do not incorporate any tunable parameters and show excellent performance in comparison to the expensive approach of the global Extended Kalman Algorithm in terms of speed of convergence and quality of solution obtained on three benchmark problems.

The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks

A general method for building and training multilayer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the structure of the network by adding units as they are needed, while a modified perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalize over patterns not in the training set.