Efficient hybrid Bayesian optimization algorithm with adaptive expected improvement acquisition function

Abstract

Computational efficiency in simulation-based optimization algorithms is essential when the system objective functions are expensive to evaluate and computational resources are limited. This article proposes a hybrid Bayesian BFGS algorithm (HB2O) to address this efficiency problem. An adaptive expected improvement (AEI) acquisition function is developed to realize a self-adaptive sampling strategy by dynamically balancing the design space exploration and exploitation. A series of computational experiments is conducted on a diverse set of test functions to benchmark the optimization performance of the HB2O against six commonly used alternative optimizers, and to validate the effectiveness of AEI against four alternative acquisition functions. The computational results show that the proposed HB2O can robustly converge on the functions’ optima with limited simulation samples, and it significantly outperforms other optimizers for various test functions. This article provides a sample-efficient solution to complex optimization problems where taking a large number of system simulations is computationally prohibitive.