Learning Algorithm For Multilayer Perceptrons Research Articles

Border Pairs Method—constructive MLP learning classification algorithm

In this paper we present the Border Pairs Method, a constructive learning algorithm for multilayer perceptron (MLP). During learning with this method a near-minimal network architecture is found. MLP learning is conducted separately by individual layers and neurons. The algorithm is tested in computer simulation with simple learning patterns (XOR and triangles image), with traditional learning patterns (Iris and Pen-Based Recognition of Handwritten Digits) and with noisy learning patterns. During the learning process we observed the following behaviour of BPM: capability to focus on global minima, good generalisation, no problems in learning with noisy, multi-dimensional and numerous learning patterns. The Border Pairs Method also supports incremental and online learning. Both are realized with or without MLP reconstruction and with or without forgetting (unlearning). The learning results with the BPM method are comparable with results from other methods.

Hybrid Optimized Back propagation Learning Algorithm for Multi-layer Perceptron

Standard neural network based on general back propagation learning using delta method or gradient descent method has some great faults like poor optimization of error-weight objective function, low learning rate, instability .This paper introduces a hybrid supervised back propagation learning algorithm which uses trust-region method of unconstrained optimization of the error objective function by using quasi-newton method .This optimization leads to more accurate weight update system for minimizing the learning error during learning phase of multi-layer perceptron.[13][14][15] In this paper augmented line search is used for finding points which satisfies Wolfe condition. In this paper, This hybrid back propagation algorithm has strong global convergence properties & is robust & efficient in practice.

A Local Supervised Learning Algorithm For Multi‐Layer Perceptrons

AbstractThe back propagation of error in multi‐layer perceptrons when used for supervised training is a non‐local algorithm in space, that is it needs the knowledge of the network topology. On the other hand, learning rules in biological systems with many hidden units, seem to be local in both space and time. In this work, a local learning algorithm is proposed which makes no distinction between input, hidden and output layers. Simulation results are presented and compared with other well known training algorithms. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

An on-line hybrid learning algorithm for multilayer perceptron in identification problems

A hybrid learning algorithm for multilayered perceptrons (MLPs) and pattern-by-pattern training, based on optimized instantaneous learning rates and the recursive least squares method, is proposed. This hybrid solution is developed for on-line identification of process models based on the use of MLPs, and can speed up the learning process of the MLPs substantially, while simultaneously preserving the stability of the learning process. For illustration and test purposes the proposed algorithm is applied to the identification of a non-linear dynamic system.

Efficiently learning multilayer perceptrons.

A learning algorithm for multilayer perceptrons is presented which is based on finding the principal components of a correlation matrix computed from the example inputs and their target outputs. For large networks our procedure needs far fewer examples to achieve good generalization than traditional on-line algorithms.

Comments on "Accelerated learning algorithm for multilayer perceptrons: optimization layer by layer"

This letter analyzes the performance of the neural network training method known as optimization layer by layer. We show, from theoretical considerations, that the amount of work required with OLL-Learning scales as the third power of the network size, compared with the square of the network size for commonly used conjugate gradient training algorithms. This theoretical estimate is confirmed through a practical example. Thus, although OLL is shown to function very well for small neural networks (less than about 500 weights per layer), it is slower than CG for large neural networks. Second, we show that OLL does not always improve on the accuracy that can be obtained with CG. It seems that the final accuracy that can be obtained depends strongly on the initial network weights.

An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer

Multilayer perceptrons are successfully used in an increasing number of nonlinear signal processing applications. The backpropagation learning algorithm, or variations hereof, is the standard method applied to the nonlinear optimization problem of adjusting the weights in the network in order to minimize a given cost function. However, backpropagation as a steepest descent approach is too slow for many applications. In this paper a new learning procedure is presented which is based on a linearization of the nonlinear processing elements and the optimization of the multilayer perceptron layer by layer. In order to limit the introduced linearization error a penalty term is added to the cost function. The new learning algorithm is applied to the problem of nonlinear prediction of chaotic time series. The proposed algorithm yields results in both accuracy and convergence rates which are orders of magnitude superior compared to conventional backpropagation learning.

Human Category Learning: Implications for Backpropagation Models

Backpropagation (Rumelhart et al., 1986) was proposed as a general learning algorithm for multi-layer perceptrons. This article demonstrates that a standard version of backprop fails to attend selectively to input dimensions in the same way as humans, suffers catastrophic forgetting of previously learned associations when novel exemplars are trained, and can be overly sensitive to linear category boundaries. Another connectionist model, ALCOVE (Kruschke 1990, 1992), does not suffer those failures. Previous researchers identified these problems; the present article reports quantitative fits of the models to new human learning data. ALCOVE can be functionally approximated by a network that uses linear-sigmoid hidden nodes, like standard backprop. It is argued that models of human category learning should incorporate quasi-local representations and dimensional attention learning, as well as error-driven learning, to address simultaneously all three phenomena.