Improvement of sep-CMA-ES for Optimization of High-Dimensional Functions with Low Effective Dimensionality

Abstract

High-dimensional black-box optimization problems often involve a property referred to as low effective dimensionality (LED), in which design variables contain many redundant elements that scarcely affect the objective function value. The covariance matrix adaptation evolution strategy (CMA-ES) suffers a performance deterioration on objective functions with LED because the redundant dimensions lead to modest hyperparameter settings and slow down the adaptation of the step-size. In this study, we focus on the separable CMA-ES (sep-CMA-ES), a variant of CMA-ES that restricts the covariance matrix to be diagonal, and propose a method to estimate the effectiveness of each dimension using the element-wise signal-to-noise ratios in the mean vector update and rank-µ update. Using the estimated effectiveness, we construct two countermeasures for LED, including an adaptation of hyperparameters and a refinement of the update rule in the cumulative step-size adaptation and two-point step-size adaptation, proposing sep-CMA-ES-LED. We experimentally showed that sep-CMA-ES-LED performed well on several benchmark functions with LED compared to the original sep-CMA-ES.