Active and Compact Entropy Search for High-Dimensional Bayesian Optimization

Abstract

Entropy search and its derivative methods are one class of Bayesian Optimization methods that achieve active exploration of black-box functions. They maximize the information gain about the position in the input space where the black-box function gets the global optimum. However, existing entropy search methods suffer from harassment caused by high dimensional optimization problems. On the one hand, the computation for estimating entropies increases exponentially as dimensions increase, which limits the applicability of entropy search to high dimensional problems. On the other hand, many high-dimensional problems have the property that a large number of dimensions have little influence on the objective function, but currently there is no compress mechanism to exclude these redundant dimensions. In this work, we propose Active Compact Entropy Search (AcCES) to fix these defects. Under the guidance of historical evaluation, AcCES actively explores the prevalent inter-dimensional correlations by maximizing the linear or non-linear relationships that may exist between dimensions in the acquisition function, which is ignored by existing Bayesian Optimization methods. In order to build a more compact input space, redundant dimensions are compressed by exploiting inter-dimensional correlations. Experiments demonstrate that AcCES achieves higher query efficiency and optimal results than existing entropy search methods.