Bayesian optimisation in unknown bounded search domains

Abstract

Bayesian optimisation (BO) is one of the most sample efficient methods for determining the optima of expensive, noisy black-box functions. Despite its tremendous success in scientific discovery and hyperparameter tuning, it still requires a bounded search space. The search spaces boundaries are, however, often chosen heuristically with an educated guess. If the boundaries are misspecified, then the search space may either be unnecessarily large and hence more expensive to optimise, or it may simply not contain the global optimum. In this paper, we introduce a method for dynamically determining the bound directly from the data. This is done using a distribution of the bound derived in a Bayesian setting. The prior is chosen by the user and the likelihood is derived with Thompson sampling. This results in a bound that is both cheap to optimise and has a high probability of containing the global optimum. We compare the performance of our method with the alternative methods on a range of synthetic and real-world problems and demonstrate that our method achieves consistently superior results.