A New Variable Step Size Adaptation Algorithm for Neural Networks

Abstract

A number of recent publications on learning systems show that adaptation rate issues are still very important research topics since practical problems require always higher and higher learning rates. It has been shown that variable step size methods can provide better convergence speed than fixed ones. This paper concentrates on variable step size methods related mainly to the different versions of the least mean square (LMS) algorithm including the backpropagation method for training feedforward neural networks, and introduces a new approach to the design of the adaptation mechanism.