New Adaptive Method to Speed Up Training of Back Propagation Algorithm

Abstract

Back Propagation (BP) algorithm considered a very effective learning approach in training of multilayer feedforward networks is. It computes the weight change to find the best weights of Artificial Neural Network (ANN). A lot of drawbacks faced the BP algorithm but the slow training and doesn't reach to local minima easily considered the main problems. A lot of algorithms were presented to improve and modify the BP algorithm through the last years. Overcoming these problems requires adding new parameters as learning rate and momentum. In this research, a new adaptive BP algorithm is proposed by introducing a new form for adaptive momentum and adaptive learning rate which depends on the error gradient at every layer. In the learning samples, the simulation results mention the convergence action of proposed algorithm. Comparing with classic BP algorithm, a better convergence rates are determined using the proposed algorithm and finds a good solution efficiently. According to the convergence rates the proposed model speed up training of BP algorithm. The accuracy of proposed algorithm reached 99.4%. The improvement in convergence rates of proposed algorithm is explained using three classification benchmark problems.