Model-form Uncertainty Research Articles

Aerodynamic Parameter Identification and Uncertainty Quantification for Small Unmanned Aircraft

The paper presents a method to identify aerodynamic parameters from flight-test data, determine the uncertainty in the identified parameters, and estimate the model-form uncertainties in the aerodynamic model. This information is required for simulations designed to estimate the reliability of the system. An output error method is used in parameter estimation, and uncertainty in these estimates is based on the Cramér–Rao bounds. Validation data are then used to compare the values of the aerodynamic coefficients as predicted by the model to experimental data. These data are used to estimate the model-form uncertainty, which is represented as prediction intervals. These methods are used to analyze both synthetic and true flight-test data. The results validate the proposed method to estimate the aerodynamic parameters and the associated model-form uncertainties. The model-form uncertainties can be directly used to predict the uncertainty in the aircraft performance and probability of loss of control.

Risk Analysis of Reactor Pressure Vessels Considering Modeling-Induced Uncertainties

Uncertainties in simulation models arise not only from the parameters that are used within the model, but also due to the modeling process itself—specifically the identification of a model that most accurately predicts the true physical response of interest. In risk-analysis studies, it is critical to consider the effect that all forms of uncertainty have on the overall level of uncertainty. This work develops an approach to quantify the effect of both parametric and model-form uncertainties. The developed approach is demonstrated on the assessment of the fatigue-based risk associated with a reactor pressure vessel subjected to a thermal shock event.

A random matrix approach for quantifying model-form uncertainties in turbulence modeling

With the ever-increasing use of Reynolds-Averaged Navier–Stokes (RANS) simulations in mission-critical applications, the quantification of model-form uncertainty in RANS models has attracted attention in the turbulence modeling community. Recently, a physics-based nonparametric approach for quantifying model-form uncertainty in RANS simulations has been proposed, where Reynolds stresses are projected to physically meaningful dimensions and perturbations are introduced only in the physically realizable limits (Xiao et al., 2016). However, a challenge associated with this approach is to assess the amount of information introduced in the prior distribution and to avoid imposing unwarranted constraints. In this work we propose a random matrix approach for quantifying model-form uncertainties in RANS simulations with the realizability of the Reynolds stress guaranteed, which is achieved by construction from the Cholesky factorization of the normalized Reynolds stress tensor. Furthermore, the maximum entropy principle is used to identify the probability distribution that satisfies the constraints from available information but without introducing artificial constraints. We demonstrate that the proposed approach is able to ensure the realizability of the Reynolds stress, albeit in a different manner from the physics-based approach. Monte Carlo sampling of the obtained probability distribution is achieved by using polynomial chaos expansion to map independent Gaussian random fields to the Reynolds stress random field with the marginal distributions and correlation structures as specified. Numerical simulations on a typical flow with separation have shown physically reasonable results, which verify the proposed approach. Therefore, the proposed method is a promising alternative to the physics-based approach for model-form uncertainty quantification of RANS simulations. The method explored in this work is general and can be extended to other complex physical systems in applied mechanics and engineering.

Adaptive Variable-Fidelity Analysis and Design Using Dynamic Fidelity Indicators

The purpose of the current study is to develop a decision-making model of a dynamic fidelity indicator (DFI) for adaptive variable-fidelity (VF) analysis and to apply it to practical engineering problems, including computational design optimization. The biggest advantage is considerable computation efficiency while satisfying solution accuracy. Bayesian-based DFI and difference-based DFI are formulated as probabilistic model validation metrics to quantify model-form uncertainties attributed to the VF analysis. Two adaptive VF surrogate models of the VF kriging with regression and sample reinterpolation and the simple VF kriging with an additive bridge function are developed. The VF kriging models are integrated with the DFIs into an efficient global optimization design framework for the design problems, which was further improved by the multipoint and multi-objective infill sampling criteria. Validation of the developed framework is carried out with two analytic functions for solution optimality and computational efficiency. As a practical design application, the NLR 7301 multi-element airfoil with a flap is designed for high lift while the model-from uncertainty is defined by different flow solutions of the Euler and Reynolds-averaged Navier–Stokes equations. The results show that the design optimization becomes more efficient with great accuracy by the adaptive VF analysis with the DFIs.

Verification and Validation in Fluids Engineering: Some Current Issues

Abstract The following current issues in verification and validation (V&V) are discussed: probability bounds analysis (PBA) versus the traditional probability distribution function (PDF) approach as used in the ANSI Standard document ASME V&V 20-2009; strict frequentist versus Bayesian approaches; model form uncertainty (and confusion) in prediction versus validation; Popper's philosophical falsificationism versus common sense; climate model trend validations versus climate change deniers; and V&V state of the art versus practice. Also, the many fundamental contributions to modern V&V practice in the history of the ASME Fluids Engineering Division and the Journal of Fluids Engineering (JFE) are described.

Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier–Stokes simulations: A data-driven, physics-informed Bayesian approach

Despite their well-known limitations, Reynolds-Averaged Navier–Stokes (RANS) models are still the workhorse tools for turbulent flow simulations in today's engineering analysis, design and optimization. While the predictive capability of RANS models depends on many factors, for many practical flows the turbulence models are by far the largest source of uncertainty. As RANS models are used in the design and safety evaluation of many mission-critical systems such as airplanes and nuclear power plants, quantifying their model-form uncertainties has significant implications in enabling risk-informed decision-making. In this work we develop a data-driven, physics-informed Bayesian framework for quantifying model-form uncertainties in RANS simulations. Uncertainties are introduced directly to the Reynolds stresses and are represented with compact parameterization accounting for empirical prior knowledge and physical constraints (e.g., realizability, smoothness, and symmetry). An iterative ensemble Kalman method is used to assimilate the prior knowledge and observation data in a Bayesian framework, and to propagate them to posterior distributions of velocities and other Quantities of Interest (QoIs). We use two representative cases, the flow over periodic hills and the flow in a square duct, to evaluate the performance of the proposed framework. Both cases are challenging for standard RANS turbulence models. Simulation results suggest that, even with very sparse observations, the obtained posterior mean velocities and other QoIs have significantly better agreement with the benchmark data compared to the baseline results. At most locations the posterior distribution adequately captures the true model error within the developed model form uncertainty bounds. The framework is a major improvement over existing black-box, physics-neutral methods for model-form uncertainty quantification, where prior knowledge and details of the models are not exploited. This approach has potential implications in many fields in which the governing equations are well understood but the model uncertainty comes from unresolved physical processes.

Quantification of uncertainties in turbulence modeling: A comparison of physics-based and random matrix theoretic approaches

Numerical models based on Reynolds-Averaged Navier-Stokes (RANS) equations are widely used in engineering turbulence modeling. However, the RANS predictions have large model-form uncertainties for many complex flows, e.g., those with non-parallel shear layers or strong mean flow curvature. Quantification of these large uncertainties originating from the modeled Reynolds stresses has attracted attention in the turbulence modeling community. Recently, a physics-based Bayesian framework for quantifying model-form uncertainties has been proposed with successful applications to several flows. Nonetheless, how to specify proper priors without introducing unwarranted, artificial information remains challenging to the current form of the physics-based approach. Another recently proposed method based on random matrix theory provides the prior distributions with maximum entropy, which is an alternative for model-form uncertainty quantification in RANS simulations. This method has better mathematical rigorousness and provides the most non-committal prior distributions without introducing artificial constraints. On the other hand, the physics-based approach has the advantages of being more flexible to incorporate available physical insights. In this work, we compare and discuss the advantages and disadvantages of the two approaches on model-form uncertainty quantification. In addition, we utilize the random matrix theoretic approach to assess and possibly improve the specification of priors used in the physics-based approach. The comparison is conducted through a test case using a canonical flow, the flow past periodic hills. The numerical results show that, to achieve maximum entropy in the prior of Reynolds stresses, the perturbations of shape parameters in Barycentric coordinates are normally distributed. Moreover, the perturbations of the turbulence kinetic energy should conform to log-normal distributions. Finally, the result sheds light on how large the variance of each physical variable should be compared with each other to achieve the approximate maximum entropy prior. The conclusion can be used as a guidance for specifying proper priors in the physics-based, Bayesian uncertainty quantification framework.

Model-Form Calibration in Drift-Diffusion Simulation Using Fractional Derivatives

In modeling and simulation, model-form uncertainty arises from the lack of knowledge and simplification during the modeling process and numerical treatment for ease of computation. Traditional uncertainty quantification (UQ) approaches are based on assumptions of stochasticity in real, reciprocal, or functional spaces to make them computationally tractable. This makes the prediction of important quantities of interest, such as rare events, difficult. In this paper, a new approach to capture model-form uncertainty is proposed. It is based on fractional calculus, and its flexibility allows us to model a family of non-Gaussian processes, which provides a more generic description of the physical world. A generalized fractional Fokker–Planck equation (fFPE) is used to describe the drift-diffusion processes under long-range correlations and memory effects. A new model-calibration approach based on the maximum mutual information is proposed to reduce model-form uncertainty, where an optimization procedure is taken.

A Bayesian Calibration–Prediction Method for Reducing Model-Form Uncertainties with Application in RANS Simulations

Model-form uncertainties in complex mechanics systems are a major obstacle for predictive simulations. Reducing these uncertainties is critical for stake-holders to make risk-informed decisions based on numerical simulations. For example, Reynolds-Averaged Navier-Stokes (RANS) simulations are increasingly used in mission-critical systems involving turbulent flows. However, for many practical flows the RANS predictions have large model-form uncertainties originating from the uncertainty in the modeled Reynolds stresses. Recently, a physics-informed Bayesian framework has been proposed to quantify and reduce model-form uncertainties in RANS simulations by utilizing sparse observation data. However, in the design stage of engineering systems, measurement data are usually not available. In the present work we extend the original framework to scenarios where there are no available data on the flow to be predicted. In the proposed method, we first calibrate the model discrepancy on a related flow with available data, leading to a statistical model for the uncertainty distribution of the Reynolds stress discrepancy. The obtained distribution is then sampled to correct the RANS-modeled Reynolds stresses for the flow to be predicted. The extended framework is a Bayesian calibration-prediction method. The merits of the proposed method are demonstrated on two flows that are challenging to standard RANS models. By not requiring observation data on the flow to be predicted, the present calibration-prediction method will gain wider acceptance in practical engineering application. While RANS modeling is chosen to demonstrate the merits of the proposed framework, the methodology is generally applicable to other complex mechanics models involving solids, fluids flows, or the coupling between the two, where model-form uncertainties are present in the constitutive relations.

Numerical strategy for model correction using physical constraints

In this paper we present a strategy for correcting model deficiency using observational data. We first present the model correction in a general form, involving both external correction and internal correction. The model correction problem is then parameterized and casted into an optimization problem, from which the parameters are determined. More importantly, we discuss the incorporation of physical constraints from the underlying physical problem. Several representative examples are presented, where the physical constraints take very different forms. Numerical tests demonstrate that the physics constrained model correction is an effective way to address model-form uncertainty.

Probability Bounds Analysis Applied to the Sandia Verification and Validation Challenge Problem

Our approach to the Sandia Verification and Validation Challenge Problem is to use probability bounds analysis (PBA) based on probabilistic representation for aleatory uncertainties and interval representation for (most) epistemic uncertainties. The nondeterministic model predictions thus take the form of p-boxes, or bounding cumulative distribution functions (CDFs) that contain all possible families of CDFs that could exist within the uncertainty bounds. The scarcity of experimental data provides little support for treatment of all uncertain inputs as purely aleatory uncertainties and also precludes significant calibration of the models. We instead seek to estimate the model form uncertainty at conditions where the experimental data are available, then extrapolate this uncertainty to conditions where no data exist. The modified area validation metric (MAVM) is employed to estimate the model form uncertainty which is important because the model involves significant simplifications (both geometric and physical nature) of the true system. The results of verification and validation processes are treated as additional interval-based uncertainties applied to the nondeterministic model predictions based on which the failure prediction is made. Based on the method employed, we estimate the probability of failure to be as large as 0.0034, concluding that the tanks are unsafe.

INCORPORATING PRIOR KNOWLEDGE FOR QUANTIFYING AND REDUCING MODEL-FORM UNCERTAINTY IN RANS SIMULATIONS

Simulations based on Reynolds-Averaged Navier--Stokes (RANS) models have been used to support high-consequence decisions related to turbulent flows. Apart from the deterministic model predictions, the decision makers are often equally concerned about the predictions confidence. Among the uncertainties in RANS simulations, the model-form uncertainty is an important or even a dominant source. Therefore, quantifying and reducing the model-form uncertainties in RANS simulations are of critical importance to make risk-informed decisions. Researchers in statistics communities have made efforts on this issue by considering numerical models as black boxes. However, this physics-neutral approach is not a most efficient use of data, and is not practical for most engineering problems. Recently, we proposed an open-box, Bayesian framework for quantifying and reducing model-form uncertainties in RANS simulations by incorporating observation data and physics-prior knowledge. It can incorporate the information from the vast body of existing empirical knowledge with mathematical rigor, which enables a more efficient usage of data. In this work, we examine the merits of incorporating various types of prior knowledge in the uncertainties quantification and reduction in RANS simulations. The result demonstrates that informative physics-based prior plays an important role in improving the quantification of model-form uncertainties, particularly when the observation data are limited. Moreover, it suggests that the proposed Bayesian framework is an effective way to incorporate empirical knowledge from various sources.

A paradigm for data-driven predictive modeling using field inversion and machine learning

We propose a modeling paradigm, termed field inversion and machine learning (FIML), that seeks to comprehensively harness data from sources such as high-fidelity simulations and experiments to aid the creation of improved closure models for computational physics applications. In contrast to inferring model parameters, this work uses inverse modeling to obtain corrective, spatially distributed functional terms, offering a route to directly address model-form errors. Once the inference has been performed over a number of problems that are representative of the deficient physics in the closure model, machine learning techniques are used to reconstruct the model corrections in terms of variables that appear in the closure model. These reconstructed functional forms are then used to augment the closure model in a predictive computational setting. As a first demonstrative example, a scalar ordinary differential equation is considered, wherein the model equation has missing and deficient terms. Following this, the methodology is extended to the prediction of turbulent channel flow. In both of these applications, the approach is demonstrated to be able to successfully reconstruct functional corrections and yield accurate predictive solutions while providing a measure of model form uncertainties.

Quantifying inflow and RANS turbulence model form uncertainties for wind engineering flows

Reynolds-averaged Navier–Stokes (RANS) simulations are often used in the wind engineering practice for the analysis of turbulent bluff body flows. An approach that allows identifying the uncertainty related to the use of reduced-order turbulence models in RANS simulations would significantly increase the confidence in the use of simulation results as a basis for design decisions. In the present study we apply a strategy that enables quantifying these uncertainties by introducing perturbations in the Reynolds stress tensor to simulations of the flow in downtown Oklahoma City. The method is combined with a framework to quantify uncertainties in the inflow wind direction and intensity, and the final result of the UQ approach is compared to field measurement data for the velocity at 13 locations in the downtown area.

Model form uncertainty versus intrinsic atomic variability in amorphous silicon oxides and nitrides

We quantify uncertainties in density functional theory predictions of several fundamental materials properties of amorphous dielectrics focusing on those that arise from the intrinsic atomic variability of the glass structures and those stemming from approximations in the theory. The intrinsic, or aleatoric, uncertainties are quantified by performing calculations over ensembles of structures obtained by annealing independent liquid samples. We estimate model form, or epistemic, uncertainties by comparing results from two exchange and correlation functionals that exhibit different bonding characteristics: the local density approximation (that typically overbinds), and the generalized gradient approximation (that often underbinds). In the case of density, bulk modulus, and point defect formation energies predictions obtained from systems containing between 72 and 192 atoms, typical of current state-of-the-art calculations, show that the intrinsic variability in the atomic structure leads to uncertainties a factor of two to four times greater than those originating from model form. While model form discrepancies remain important, our results emphasize the importance of using ensembles of structures to make predictions of amorphous materials. The use of such probabilistic atomic-level data as input in multiscale materials or device models is critical for predictions with quantified uncertainties but also to uncover how atomic variability affects device performance.

Automatic product conceptual optimization based on object-oriented performance components and graph grammars

Compared with parameter uncertainty, model form uncertainty is difficult and expensive to alleviate due to limited design information at the early product development stage, but it may have significant impacts on performances of complex products. In this article, an optimal configuration of performance models based on graph grammars is presented to seek a solution for simultaneously resolving model form and parameter uncertainties at the product conceptual design stage. First, a hybrid performance component database is built using object-oriented modeling method taking account of the model evolution history; second, a function structure model is taken as the reference template to dynamically generate system models driven by graph grammars rules derived from product configuration and layout design and matching of component interface parameters with flows in the function structure model; third, a meta-model-based method is adopted to transform the performance component graph into mathematical models including the optimization objective function and associated constraints; fourth, an optimization procedure combined with performance simulation framework is invoked to solve the optimization problem. The final result is a solution that includes the optimal configuration of performance components and optimal design parameters, and thus, much larger design space than traditional mere optimization of design variables is explored. The method is tested by the optimal conceptual design of tank movement systems, and comparisons of deign results obtained by the proposed method and traditional optimization method are conducted to illustrate the efficacy of the method.

A case study to quantify prediction bounds caused by model-form uncertainty of a portal frame

Numerical simulations, irrespective of the discipline or application, are often plagued by arbitrary numerical and modeling choices. Arbitrary choices can originate from kinematic assumptions, for example the use of 1D beam, 2D shell, or 3D continuum elements, mesh discretization choices, boundary condition models, and the representation of contact and friction in the simulation. This work takes a step toward understanding the effect of arbitrary choices and model-form assumptions on the accuracy of numerical predictions. The application is the simulation of the first four resonant frequencies of a one-story aluminum portal frame structure under free–free boundary conditions. The main challenge of the portal frame structure resides in modeling the joint connections, for which different modeling assumptions are available. To study this model-form uncertainty, and compare it to other types of uncertainty, two finite element models are developed using solid elements, and with differing representations of the beam-to-column and column-to-base plate connections: (i) contact stiffness coefficients or (ii) tied nodes. Test-analysis correlation is performed to compare the lower and upper bounds of numerical predictions obtained from parametric studies of the joint modeling strategies to the range of experimentally obtained natural frequencies. The approach proposed is, first, to characterize the experimental variability of the joints by varying the bolt torque, method of bolt tightening, and the sequence in which the bolts are tightened. The second step is to convert what is learned from these experimental studies to models that “envelope” the range of observed bolt behavior. We show that this approach, that combines small-scale experiments, sensitivity analysis studies, and bounding-case models, successfully produces lower and upper bounds of resonant frequency predictions that match those measured experimentally on the frame structure. (Approved for unlimited, public release, LA-UR-13–27561).

Quantification of model form uncertainty in the calculation of solar diffuse irradiation on inclined surfaces for building energy simulation

Traditional uncertainty quantification (UQ) in the prediction of building energy consumption has been limited to the propagation of uncertainties in model input parameters. Models by definition ignore, at least to some degree, and, in almost all cases, simplify the physical processes that govern the reality of interest, thereby introducing additional uncertainty in model predictions that cannot be captured as input parameter uncertainty. Quantification of this type of uncertainty (which we will refer to as model form uncertainty) is a necessary step towards the complete UQ of model predictions. This paper introduces a general framework for model form UQ and shows its application to the widely used sky irradiation model developed by Perez et al. [1990. “Modeling Daylight Availability and Irradiance Components from Direct and Global Irradiance.” Solar Energy 44 (5): 271–289], which computes solar diffuse irradiation on inclined surfaces. We collected a data set of one-year measurements of solar irradiation at one location in the USA. The measurements were done at surfaces with different tilt angles and orientations, for a wide spectrum of sky conditions. A statistical analysis using both this data set and published studies worldwide suggests that the Perez model performs non-uniformly across different locations and produces a certain bias in its predictions. Based on the same data, we then use a two-phase regression model to express model form uncertainty in the use of the Perez model at this particular location. Using a holdout validation test, we demonstrate that the two-phase regression model considerably reduces the model bias errors and root mean square errors for every tilted surface. Lastly, we discuss the significance of including model form uncertainty in the energy consumption predictions obtained with whole building simulation.

A Bayesian statistical method for quantifying model form uncertainty and two model combination methods

Apart from parametric uncertainty, model form uncertainty as well as prediction error may be involved in the analysis of engineering system. Model form uncertainty, inherently existing in selecting the best approximation from a model set cannot be ignored, especially when the predictions by competing models show significant differences. In this research, a methodology based on maximum likelihood estimation is presented to quantify model form uncertainty using the measured differences of experimental and model outcomes, and is compared with a fully Bayesian estimation to demonstrate its effectiveness. While a method called the adjustment factor approach is utilized to propagate model form uncertainty alone into the prediction of a system response, a method called model averaging is utilized to incorporate both model form uncertainty and prediction error into it. A numerical problem of concrete creep is used to demonstrate the processes for quantifying model form uncertainty and implementing the adjustment factor approach and model averaging. Finally, the presented methodology is applied to characterize the engineering benefits of a laser peening process.

The deviation from parallel shear flow as an indicator of linear eddy-viscosity model inaccuracy

A marker function designed to indicate in which regions of a generic flow field the results from linear eddy-viscosity turbulence models are plausibly inaccurate is introduced. The marker is defined to identify regions that deviate from parallel shear flow. For two different flow fields it is shown that these regions largely coincide with regions where the prediction of the Reynolds stress divergence is inaccurate. The marker therefore offers a guideline for interpreting results obtained from Reynolds-averaged Navier-Stokes simulations and provides a basis for the further development of turbulence model-form uncertainty quantification methods.