Deep Residual Surrogate Model

Abstract

Surrogate models are widely used to model the high computational cost problems such as industrial simulation or engineering optimization when the size of sampled data for modeling is greatly limited. They can significantly improve the efficiency of complex calculations by modeling original expensive problems with simpler computation-saving functions. However, a single surrogate model cannot always perform well for various problems. On this occasion, hybrid surrogate models are created to improve the final performances on different problems by combining advantages of multiple single models. Nevertheless, existing hybrid methods work by estimating weights for all alternative single models, which limits the efficiency when more single models are adopted. In this paper, we propose a novel hybrid surrogate model quite different from former methods, named the Deep Residual Surrogate model (DRS). DRS does not merge all alternative single surrogate models directly by weights, but by assembling selected ones in a multiple layers structure. We propose first derivate validation (FDV) to recurrently select the single surrogate model adopted in each layer from all candidates. Experimental results on multiple benchmark problems demonstrate that DRS has better performances than existing single and hybrid surrogate models in both prediction accuracy and stability with higher efficiency.