Impact of Categorical Variable Encodings on Bayesian Optimization Performance

Abstract

A large variety of engineering problems can be solved using black-box optimization techniques. For the optimization of very costly black-box experiments and simulations, surrogate model-based optimization (SMBO) methods like Bayesian Optimization (BO) are applied increasingly often. Engineering design tasks commonly include categorical variables, for which corresponding variable encodings must be chosen. However, the consequences of this choice are often unclear and receive little attention. In this contribution, we specifically focus on how categorical variable encodings define the shape of black-box functions and thereby substantially influence the performance of BO. We introduce a spatial entropy metric to quantify structural differences of function permutations induced by different variable encodings. This metric allows explaining BO performances on toy functions, which deteriorate with an increasing disorder in search spaces caused by poor variable encodings. In addition, we present a BO experiment of an electronic design example, for which we analyze the performance of physics-based variable encodings and compare it to physics neglecting encodings. Results show that utilizing physics-based encodings is crucial for achieving fast optimization convergences typically associated with BO.