A backpropagation algorithm with adaptive learning rate and momentum coefficient

Abstract

Slower convergence and longer training times are the disadvantages often mentioned when the conventional backpropagation (BP) algorithm are compared with other competing techniques. In addition, in the conventional BP algorithm, the learning rate is fixed and that it is uniform for all the weights in a layer. In this paper, we propose an efficient acceleration technique, the backpropagation with adaptive learning rate and momentum term, which is based on the conventional BP algorithm by employing an adaptive learning rate and momentum factor, where the learning rate and momentum rate are adjusted at each iteration to reduce the training time. Simulation results indicate a superior convergence speed as compared to other competing methods.