A Constructive Approach of Modified Standard Backpropagation Algorithm with Optimum Initialization for Feedforward Neural Networks

Abstract

For the learning process of multilayer feedforward neural network, the constructive approach of Modified Backpropagation algorithm (MBP), with optimum initialization is proposed. One of the common complaints about the Standard Backpropagation algorithm (SBP) is that it is very slow. Even simple problems may take hundreds of iterations to converge. SBP algorithm reduces only non-linear errors. Much work, therefore has been done in search of faster methods. One of such approach is modified form of the Standard Backpropagation algorithm. Modified Backpropagation algorithm consists of minimizing the sum of the squares of linear and non- linear errors for all output units. This leads to an efficient process in the network. Proper initialization always plays a key role in the robust neural networks. Therefore, the optimum initialization method is used for weight initialization, which ensures the outputs of neurons are in the active region and the range of activation function is fully utilized. Since the proposed method uses the constructive approach, there is no need to make a prior estimate of the correct network size. The proposed method is implemented on 2 bit parity problem, 4 bit parity checker and encoder problem and produced good results.