Evaluation of Model Validation Techniques in the Presence of Uncertainty

Abstract

Model validation is the assessment of the correctness of a model relative to experimental data. The results of a model validation study can be used to either quantify the model form uncertainty or to improve the model (i.e., calibration). The model validation process is generally complicated by the fact that both the simulation outcomes and the experimental outcomes include significant uncertainty which can come in the form of random (aleatory) uncertainties, lack-of-knowledge (epistemic) uncertainties, and bias errors. In this paper, we examine four different approaches to model validation: 1) the area validation metric, 2) the standard validation uncertainty, 3) a modified area validation metric, and 4) an approach based on comparison on probability density functions. In order to provide a rigorous assessment of these model validation frameworks, we employ the recently developed Method of Manufactured Universes (MMU). The main advantage of MMU is that the “true” value in nature is known, thus allowing a rigorous assessment of whether the model validation approaches provide conservative and tight bounds on the true model error. MMU is applied to the compressible turbulent flow over a NACA 0012 airfoil. The “true” values in the manufactured universe are found using turbulent computational fluid dynamics simulations while the “model” employs simplified lift and drag estimates based on thin airfoil theory and empirical drag correlations. Preliminary results indicate that the area validation metrics are less susceptible to over-predict model form uncertainty and that the modified area validation metric further tightens the bounds on the true model error relative to the original area validation metric.