The importance of an adaptive gain variation method in back-propagation neural network learning algorithms

Abstract

In a 'feed forward' algorithm, the slope of the activation function is directly influenced by a parameter referred to as 'gain'. In this paper, the influence of the variation of 'gain' on the learning ability of a neural network is analysed. Multi layer feed forward neural networks have been assessed. Physical interpretation of the relationship between the gain value and learning rate and weight values is given. Instead of a constant 'gain' value, we propose an algorithm to change the gain value adaptively for each node. The efficacy of the proposed method is verified by means of simulation on a function approximation problem using sequential mode of training. The results show that the proposed method considerably improves the learning speed of the general back-propagation algorithm.