Comparative studies of error metrics in variable fidelity model uncertainty quantification

Abstract

ABSTRACTVariable-fidelity (VF) surrogate models which integrate different fidelities of date are wildly used in simulation-based modelling due to a good balance of modelling expense and modelling accuracy. However, VF surrogate models built by limited sample points inevitably have large prediction uncertainty. Using inaccurate VF models in the design and optimisation process may lead to distort predictions or optimal solutions that locate in unfeasible region. Besides, if inappropriate error metrics are utilised in the uncertainty quantifying of a surrogate model, misleading or erroneous evaluation results will be obtained, which may lead to the wrong usage of it in design process. In this paper, the performance of four error metrics (bootstrap error, leave-one-out (LOO) error, mean square error (MSE) and predictive estimation of model fidelity (PEMF error) is systematically compared in uncertainty quantification of VF surrogate model. A set of numerical examples with different features and a long cylinder pressure vessel design problem are utilised to test the performance of the error metrics. The error metrics are evaluated from different aspects, including the number of sample points, sampling methods, and dimension of the test problems etc. Results show that in low dimensional problems, MSE shows excellent error prediction capability not only in efficiency but also in effectiveness while LOO error performs the best in high dimensional problems. Based on the comparison results, a useful guideline for selecting the most appropriate error metric for the problems with different characteristics is provided.