Non-inertial opposition-based particle swarm optimization and its theoretical analysis for deep learning applications

Abstract

Particle Swarm Optimization (PSO) and its variants are often used to train and optimize the structure and parameters of deep learning models to improve accuracy of learning results in the reasonable time-consuming. The performance of PSO is completed determined by its kinetic equation of particles. To accelerate convergence rate, a novel kinetic equation without inertial term is devised and applied to PSO, and then a non-inertial opposition-based particle swarm optimization (NOPSO) is generated combined with a adaptive elite mutation strategy and generalized opposition-based learning strategy. Simulation Experimental results show that the new kinetic equation has effectively accelerated convergence rate of PSO. Meanwhile, Theoretical analysis of the new kinetic equation is carried out by order-2 difference recurrence equation, the inference conclusions of which are consistent with the results of simulation experiments. NOPSO algorithm with a new kinetic equation is a highly competitive algorithm compared with some state-of-art PSOs and is suitable for deep learning applications.