H/sub ∞/-learning of layered neural networks

Abstract

Although the backpropagation (BP) scheme is widely used as a learning algorithm for multilayered neural networks, the learning speed of the BP algorithm to obtain acceptable errors is unsatisfactory in spite of some improvements such as introduction of a momentum factor and an adaptive learning rate in the weight adjustment. To solve this problem, a fast learning algorithm based on the extended Kalman filter (EKF) is presented and fortunately its computational complexity has been reduced by some simplifications. In general, however, the Kalman filtering algorithm is well known to be sensitive to the nature of noises which is generally assumed to be Gaussian. In addition, the H/sub /spl infin// theory suggests that the maximum energy gain of the Kalman algorithm from disturbances to the estimation error has no upper bound. Therefore, the EKF-based learning algorithms should be improved to enhance the robustness to variations in the initial values of link weights and thresholds as well as to the nature of noises. The paper proposes H/sub /spl infin//-learning as a novel learning rule and to derive new globally and locally optimized learning algorithms based on H/sub /spl infin//-learning. Their learning behavior is analyzed from various points of view using computer simulations. The derived algorithms are also compared, in performance and computational cost, with the conventional BP and EKF learning algorithms.