A reinforcement learning-based neighborhood search operator for multi-modal optimization and its applications

Abstract

In this paper, a reinforcement learning-based neighborhood search operator (RLNS) is proposed for multi-modal optimization problems where the main novelties lie in the reinforcement learning-based neighborhood range selection strategy, the neighborhood subpopulation generation strategy and the local vector encirclement model. The reinforcement learning-based neighborhood range selection strategy is proposed to dynamically adjust the subpopulation size to address the issue of too many parameters to be adjusted in the multi-modal optimization algorithm based on the niching methods, while the neighborhood subpopulation generation strategy and the local vector encirclement model are designed with the hope of enhancing the individual’s ability to local exploitation to obtain more accurate solutions. To verify the effectiveness of the proposed RLNS, SSA-RLNS, PSO-RLNS and EO-RLNS are proposed by integrating the proposed RLNS with the existing sparrow search algorithm, particle swarm optimization and equilibrium optimizer. The performances of the proposed SSA-RLNS, PSO-RLNS, EO-RLNS and existing multi-modal optimization algorithms are tested in CEC2015 multi-niche benchmark functions. The experimental results show that the SSA-RLNS, PSO-RLNS and EO-RLNS could locate multiple global optimal solutions with satisfactory accuracy, which illustrate that the proposed RLNS could be successfully used to deal with multi-modal optimization problems by integrating with common population-based optimization algorithms. Finally, the SSA-RLNS is successfully applied in the inverse kinematics of robot manipulator.