Supporting Hyperparameter Optimization in Adaptive Sampling Methods

Abstract

Abstract Surrogate models can be used to reduce the computational load when a simulation model is computationally costly to evaluate. This is the case if sophisticated thermodynamic models are integrated as e.g. the Perturbed-Chain Statistical Associating Fluid Theory (PC-SAFT) equation of state. When constructing surrogate models, the question of how to choose the training set arises. Recent research showed that promising results were obtained using adaptive or sequential sampling methods. In these approaches, the surrogate model predictions are used to identify additional promising sample locations. The results depend on the structure of the surrogate model, i.e. the choice of the hyperparameters. It is in general a tedious task to choose hyperparameters by trial and error, and a set of hyperparameters that is suitable in the initial phase may not be adequate anymore when the size of the training set increases significantly. Therefore, we here propose a methodology to incorporate hyperparameter optimization (HPO) into the adaptive sampling workflow. As this comes with a significant effort, HPO is only performed when it promises improvements.