Gradient-Enhanced Kriging for High-Dimensional Bayesian Optimization with Linear Embedding

Abstract

This paper explores the application of gradient-enhanced (GE) kriging for Bayesian optimization (BO) problems with a high-dimensional parameter space. We utilize the active subspace method to embed the original parameter space in a low-dimensional subspace. The active subspace is detected by analyzing the spectrum of the empirical second-moment matrix of the gradients of the response function. By mapping the training data onto their respective subspace, the objective function and the constraint functions are efficiently approximated with low-dimensional GE-kriging models. In each cycle of the BO procedure, a new point is found by maximizing the constrained expected improvement function within a low-dimensional polytope, and it is mapped back to the original space for model evaluation. In this way, the computational costs are significantly reduced when compared with standard GE-kriging. We illustrate and assess the proposed approach with numerical experiments ranging from academic benchmark problems to aerodynamic engineering applications. The experimental results show that the proposed method is promising for optimizing high-dimensional expensive objective functions, especially for problems that exhibit a clear low-dimensional active subspace.