A Combined Training Algorithm for RBF Neural Network Based on Particle Swarm Optimization and Gradient Descent

Abstract

Radial basis function neural network usually has an extremely complex surface of the error function for its strong non-linear mapping capability. This brings us to lot of challenges for the neural network training, especially when you can’t find a suitable training method. The convergence rate of radial basis function neural network will be slow when the gradient descent algorithm is used for the training of neural network. Meanwhile, it also has a great possibility to fall into the local minimum which means the network may performs badly in the real prediction. On the other hand, the training method based on particle swarm optimization algorithm shows the weakness on local searching ability, although it can get rid of the trouble of falling into the local minimum than the gradient descent algorithm to some extent. In order to solve the above problems, on the basis of the characteristics of the two algorithms, this paper proposes a combined training algorithm for radial basis function neural network, which can overcome the disadvantage of the above algorithm and take advantage of the two in the meanwhile. In the combined algorithm, the neural network is trained by the particle swarm optimization algorithm at first. As the training progresses, we replace particle swarm optimization algorithm with the gradient descent algorithm after the threshold of convergence rate is reached. Besides, some data sets in UCI are simulated with the proposed training method. The performance of proposed training algorithm is validated by experimental result and compared to other training algorithm.