A decomposition-based many-objective evolutionary algorithm with Q-learning guide weight vectors update

Abstract

When dealing with regular, simple Pareto fronts (PFs), the decomposition-based multi-objective optimization algorithm (MOEA/D) performs well by presetting a set of uniformly distributed weight vectors. However, its performance declines when faced with complex and irregular PFs. Many algorithms address this problem by periodically adjusting the distribution of the weight vectors, but these methods do not take into account the performance of the population and are likely to update the weight vectors at the wrong time. In addition, for the SBX crossover operator, the setting of its distribution index will largely affect the exploration and convergence ability of the algorithm, so a single parameter setting will have negative impacts. To tackle these challenges, this paper proposes a method to simultaneously adaptively update weight vectors and optimize SBX parameter via Q-learning(RL-MaOEA/D). In order to make the strategies made by Q-learning more accurate, Two different metrics (CD and NCD) are proposed that capture diversity and convergence of individual and population respectively. RL-MaOEA/D is compared with seven state-of-the-art algorithms on different problems, and the simulation results reflect that the proposed algorithm has better performance.