Deep reinforcement learning assisted novelty search in Voronoi regions for constrained multi-objective optimization

Abstract

Solving constrained multi-objective optimization problems (CMOPs) requires optimizing multiple conflicting objectives while satisfying various constraints. Existing constrained multi-objective evolutionary algorithms (CMOEAs) cross infeasible regions by ignoring constraints. However, these methods might neglect promising search directions, leading to insufficient exploration of the search space. To address this issue, this paper proposes a deep reinforcement learning assisted constrained multi-objective quality-diversity algorithm. The proposed algorithm designs a diversity maintenance mechanism to promote evenly coverage of the final solution set on the constrained Pareto front. Specifically, first, a novelty-oriented archive is created using a centroid Voronoi tessellation, which divides the search space into a desired number of Voronoi regions. Each region acts as a repository of non-dominated solutions with different phenotypic characteristics to provide diversity information and supplementary evolutionary trails. Secondly, to improve resource utilization, a deep Q-network is adopted to learn a policy to select suitable Voronoi regions for offspring generation based on their novelty scores. The exploration of these regions aims to find a set of diverse, high-performing solutions to accelerate convergence and escape local optima. Compared with eight state-of-the-art CMOEAs, experimental studies on four benchmark suites and nine real-world applications demonstrate that the proposed algorithm exhibits superior or at least competitive performance, especially on problems with discrete and narrow feasible regions.