Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

Abstract

Bayesian optimization (BO) is a widely used data-driven method for the global optimization of black-box objective functions with noise. A key component of BO is surrogate model that is a probabilistic model used for approximating black-box objective functions. However, classical BO methods often utilize Gaussian processes with stationary covariance functions as surrogate models, which is difficult to accurately model complex systems with rapidly oscillating target functions, and therefore deteriorates the performance of BO. To this end, a Bayesian optimization approach based on deep kernel learning (DKL) is investigated for the optimization of complex objective functions with large oscillations, in which a Gaussian process based on deep kernel learning is utilized to capture the characteristics of data with non-stationary and hierarchical covariance functions via deep neural networks. The optimization performance of the DKL-based BO approach with different network structures is quantified and compared with a state-of-the-art BO method based on Gaussian process under the scenarios of different noise levels.