Hierarchical Gaussian Process Models for Improved Metamodeling

Abstract

Simulations are often used for the design of complex systems as they allow one to explore the design space without the need to build several prototypes. Over the years, the simulation accuracy, as well as the associated computational cost, has increased significantly, limiting the overall number of simulations during the design process. Therefore, metamodeling aims to approximate the simulation response with a cheap to evaluate mathematical approximation, learned from a limited set of simulator evaluations. Kernel-based methods using stationary kernels are nowadays widely used. In many problems, the smoothness of the function varies in space, which we call nonstationary behavior [20]. However, using stationary kernels for nonstationary responses can be inappropriate and result in poor models when combined with sequential design. We present the application of two recent techniques: Deep Gaussian Processes and Gaussian Processes with nonstationary kernel, which are better able to cope with these difficulties. We evaluate the method for nonstationary regression on a series of real-world problems, showing that these recent approaches outperform the standard Gaussian Processes with stationary kernels. Results show that these techniques are suitable for the simulation community, and we outline the variational inference method for the Gaussian Process with nonstationary kernel.