Cooperative Bayesian optimization with hybrid grouping strategy and sample transfer for expensive large-scale black-box problems

Abstract

Expensive large-scale optimization problems are difficult to solve owing to their high complexity and computational costs. Bayesian optimization is an effective method for dealing with low-dimensional, costly problems. This study extends Bayesian optimization to expensive large-scale optimization problems. In the study, the large-scale problem is decomposed into a series of low-dimensional subproblems using the proposed hybrid grouping strategy. Each subproblem is chosen based on its contribution to the improvement of the best overall objective value, and then Bayesian optimization is used to solve it. This study proposes a sample transfer strategy and a novel transfer Gaussian process regression model to improve the efficiency of Bayesian optimization. The sample transfer strategy can take advantage of the additive separability of the objective and thus transfers the historical samples by adding a constant to their objective values. The transfer Gaussian process regression model can employ the historical samples by reducing their correlations according to the differences in their parameters (i.e., variables other than those of the current subproblem). The proposed algorithm is tested on 15 large-scale benchmark problems up to 1000 dimensions. The experimental results show that the proposed algorithm outperforms the state-of-the-art algorithms, and the proposed hybrid grouping strategy, sample transfer strategy, and transfer Gaussian process regression improve the efficiency of the cooperative Bayesian optimization framework even further.