An adaptive conjugate gradient learning algorithm for efficient training of neural networks

Abstract

An adaptive conjugate gradient learning algorithm has been developed for training of multilayer feedforward neural networks. The problem of arbitrary trial-and-error selection of the learning and momentum ratios encountered in the momentum backpropagation algorithm is circumvented in the new adaptive algorithm. Instead of constant learning and momentum ratios, the step length in the inexact line search is adapted during the learning process through a mathematical approach. Thus, the new adaptive algorithm provides a more solid mathematical foundation for neural network learning. The algorithm has been implemented in C on a SUN-SPARCstation and applied to two different domains: engineering design and image recognition. It is shown that the adaptive neural networks algorithm has superior convergence property compared with the momentum backpropagation algorithm.