Hybrid hyperplane gradient learning algorithm for RBF neural network

Abstract

Gradient-based algorithms are still popularly used for training radial basis function neural network (RBFNN). However, these algorithms may lead to the vanishing gradient (saddle point or local minimum) problem, which is one of the common problems that limit learning performance. To cope with this, a hybrid hyperplane gradient learning algorithm (HHGLA) is proposed to improve the learning performance of the RBFNN in this paper. First, a hyperplane gradient (HPG), based on a hyperspace constructed by a solution population (SP), is introduced. Thus, the search process can cross the surface of the cost function to avoid the appearance of vanishing gradient. Second, an adaptive learning rate is designed to restrict the search process into this hyperspace. Then, during the learning process, the hyperspace is constantly contracted to approximate the global optimal solution. Third, the convergence property of HHGLA-based RBFNN is thoroughly analyzed to ensure its application success. Finally, empirical comparisons of the proposed HHGLA-RBFNN are given to illustrate its superiority. The simulation results show the feasibility of HHGLA-RBFNN of accuracy.