A trust region based local Bayesian optimization without exhausted optimization of acquisition function

Abstract

Bayesian optimization (BO) is an effective optimization technique for solving expensive black-box problems. Even though BO has remarkable success, its drawbacks are also obvious. First, the time complexity of the Gaussian process inference is higher than O(n3), where n is the number of samples. Consequently, the running time of BO increases rapidly with the problem size. Second, due to the non-convexity and multimodality of the acquisition function, it costs a lot to achieve good results. To address the above problems, we develop a local Bayesian optimization algorithm based on the trust region idea (TRLBO). In TRLBO, two trust regions with dynamically changing sizes are used to enhance the algorithm’s exploitation ability, while at the same time retaining the exploration ability. Specifically, one trust region is used to reduce the number of samples in the Gaussian process. The other is used to restrict the solution space of the candidates. Furthermore, some theoretical results were provided to enlighten the efficiency of the proposed algorithm. Experimental results on both benchmark functions and real-world problems show that TRLBO compares favorably with the state-of-the-art algorithms.