Abstract
The feedforward neural network architecture uses backpropagation learning to determine optimal weights between different interconnected layers. This learning procedure uses a gradient descent technique applied to a sum-of-squares error function for the given input-output pattern. It employs an iterative procedure to minimise the error function for a given set of patterns, by adjusting the weights of the network. The first derivates of the error with respect to the weights identify the local error surface in the descent direction. Hence the network exhibits a different local error surface for every different pattern presented to it, and weights are iteratively modified in order to minimise the current local error. The determination of an optimal weight vector is possible only when the total minimum error (mean of the minimum local errors) for all patterns from the training set may be minimised. In this paper, we present a general mathematical formulation for the second derivative of the error function with respect to the weights (which represents a conjugate descent) for arbitrary feedforward neural network topologies, and we use this derivative information to obtain the optimal weight vector. The local error is backpropagated among the units of hidden layers via the second order derivative of the error with respect to the weights of the hidden and output layers independently and also in combination. The new total minimum error point may be evaluated with the help of the current total minimum error and the current minimised local error. The weight modification processes is performed twice: once with respect to the present local error and once more with respect to the current total or mean error. We present some numerical evidence that our proposed method yields better network weights than those determined via a conventional gradient descent approach.
Highlights
In the feedforward neural network architecture, the algorithm for modifying the weights between different interconnected layers is usually known as the backpropagation learning technique [19]
The incorporation of second order derivative information of the error function results in finding an optimal weight vector corresponding to different local error surfaces, all the while attempting to minimise the cumulative error corresponding to all the input-output pattern pairs
In order to analyze the performance of the feedforward neural network with conjugate descent for the character recognition problem, the parameter values in Table 1 were used in all the experiments
Summary
In the feedforward neural network architecture, the algorithm for modifying the weights between different interconnected layers is usually known as the backpropagation learning technique [19] This algorithm evaluates the first derivative of the error function with. The term backpropagation is used to describe the training of a multilayer neural network using a gradient descent approach applied to a sum-of-squares error function for the given pattern. The proposed formulation provides a general way to obtain optimal weights from the minimisation of the total or mean error which consists of combining available minimised local errors To accomplish this we realize second order derivatives of the error and find weight modifications in order to obtain the minimum point of every unknown instantaneous local error. We provide numerical evidence which supports this observation of superior convergence of the conjugate descent method over a standard backpropagation approach
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
