Improved Gaussian Process Acquisition for Targeted Bayesian Optimization

Abstract

A black-box optimization problem is considered, in which the function to be optimized can only be expressed in terms of a complicated stochastic algorithm that takes a long time to evaluate. The value returned is required to be sufficiently near to a target value, and uses data that has a significant noise component. Bayesian Optimization with an underlying Gaussian Process is used as an optimization solution, and its effectiveness is measured in terms of the number of function evaluations required to attain the target. To improve results, a simple modification of the Gaussian Process ‘Lower Confidence Bound’ acquisition function is proposed. With this change, much improved results compared to random selection methods and to other commonly used acquisition functions are obtained. Additional modifications result in further improvement. An intuitive explanation and a formal proof are given to explain the reduction in the number of lengthy function evaluations needed to reach the target.