Uncertainty Quantification Procedure Research Articles

Non-probabilistic credible set model for structural uncertainty quantification

A non-probabilistic credible set model for uncertainty quantification structural parameters is proposed in this paper. The developed method surpasses the probabilistic uncertainty quantification method and traditional non-probabilistic uncertainty quantification method by providing both the distribution range of uncertain structural parameters and the corresponding credibility for scant samples, thereby reducing the computation time and improving the credibility of the uncertainty quantification process. The average inverse-distance information entropy, the expanded uncertainty and credibility are introduced to establish a one-dimensional non-probabilistic credible set uncertainty quantification procedure. The basic theory of non-probabilistic sets and a correlation analysis of multidimensional uncertain parameters are reviewed. This theory and analysis are used to develop a multidimensional non-probabilistic credible set uncertainty quantification method. The optimal non-probabilistic set is searched to determine the local coordinate system. The multidimensional uncertainty quantification is subsequently transformed to a one-dimensional uncertainty quantification. A civil aircraft wing structure case, a stiffened plate case and a dynamic case are utilized to verify the feasibility of applying the developed method to various engineering problems.

Development of an aqueous lignin mixture thermophysical model for hydrothermal liquefaction applications using uncertainty quantification tools

A thermophysical model is developed that can predict the properties of two lignin mixtures, black liquor and lignosulfonates, up to 50% mass fractions, at hydrothermal conditions. An uncertainty quantification framework linked with classic thermodynamical modelling was included to account for the extreme variability of the raw material. An idealised flow simulation verified the model, where hot compressed water mixes with a cold, aqueous lignin stream in a T-piece reactor configuration. The uncertainty quantification procedure determined that density and heat capacity uncertainty significantly influence residence time, and viscosity uncertainty mainly affects mixing. Micromixing time is fivefold and ten-fold higher for black liquor and lignosulfonates mixtures, respectively, compared to pure water mixing. The uncertainty in all simulated quantities of interest caused by the thermophysical model is reduced by increasing flow rates. This study predicted chemical reactor behaviour under varying thermophysical conditions and their final effect in terms of confidence intervals.

Hierarchical Bayesian learning framework for multi-level modeling using multi-level data

A hierarchical Bayesian learning framework is proposed to account for multi-level modeling in structural dynamics. In multi-level modeling the system is considered as a hierarchy of lower-level models, starting at the lowest material level, progressing to the component level, then the subsystem level, before ending up to the system level. Bayesian modeling and uncertainty quantification techniques based on measurements that rely on data collected only at the system level cover a quite limited number of component/subsystem operating conditions that are far from representing the full spectrum of system operating conditions. In addition, the large number of models and parameters involved from the lower to higher modeling levels of the system, constitutes this approach inappropriate for simultaneously and reliably quantifying the uncertainties at the different modeling levels. In this work, comprehensive hierarchical Bayesian learning tools are proposed to account for uncertainties through the multi-level modeling process. The uncertainty is embedded within the structural model parameters by introducing a probability model for these parameters that depend on hyperparameters. An important issue that has to be accounted for is that parameters of models at lower levels are shared at the subsystem and system levels. This necessitates a parameter inference process that takes into account data from different modeling levels. Accurate and insightful asymptotic approximations are developed, substantially reducing the computational effort required in the parameter uncertainty quantification procedure. The uncertainties inferred based on datasets available from the different levels of model hierarchy are propagated through the different levels of the system to predict uncertainties and confidence levels of output quantities of interest. A simple dynamical system consisting of components and subsystems is employed to demonstrate the effectiveness of the proposed method.

Statistical prediction of waterflooding performance by K-means clustering and empirical modeling

Statistical prediction is often required in reservoir simulation to quantify production uncertainty or assess potential risks. Most existing uncertainty quantification procedures aim to decompose the input random field to independent random variables, and may suffer from the curse of dimensionality if the correlation scale is small compared to the domain size. In this work, we develop and test a new approach, K-means clustering assisted empirical modeling, for efficiently estimating waterflooding performance for multiple geological realizations. This method performs single-phase flow simulations in a large number of realizations, and uses K-means clustering to select only a few representatives, on which the two-phase flow simulations are implemented. The empirical models are then adopted to describe the relation between the single-phase solutions and the two-phase solutions using these representatives. Finally, the two-phase solutions in all realizations can be predicted using the empirical models readily. The method is applied to both 2D and 3D synthetic models and is shown to perform well in the P10, P50 and P90 of production rates, as well as the probability distributions as illustrated by cumulative density functions. It is able to capture the ensemble statistics of the Monte Carlo simulation results with a large number of realizations, and the computational cost is significantly reduced.

Towards application of uncertainty quantification procedure combined with experimental procedure for assessment of the accuracy of the DEM approach dedicated for granular flow modeling

There is a high demand for accurate and fast numerical models for dense granular flows found in many industrial applications. Nevertheless, before numerical model can be used its need to be always validated against experimental data. During the validation, it is important to consider how the measurement data sets, as well as the numerical models, are affected by errors and uncertainties. In this study, the uncertainty quantification for the Discrete Element Method (DEM) model was performed based on selected quantities of interest (QOI), which were measured at a test rig. The uncertainty quantification was performed with open-source Dakota code, and the Latin hypercube sampling technique was used to determine test points. Various correlations between the input and output data were investigated to assess the impact of the possible input data errors on the output values determined by the solver. The results were validated against the measurement data from a novel in-house experimental test rig. The novel character of this work is developed procedure for study the impact of the uncertainties related with the input data on numerical results delivered by DEM model. In-house algorithm written using OpenCV libraries for determining particle motion characteristics from image dataset was used for results determination and postprocessing.

Uncertainty quantification and sensitivity analysis of aerothermal performance for the turbine blade squealer tip

In this paper, an uncertainty quantification (UQ) method is proposed using the non-intrusive polynomial chaos expansion method and Smolyak sparse grids. Then coupled with three-dimensional Reynolds-Averaged Navier-Stokes (RANS) solutions, an uncertainty quantification procedure is carried out for aerodynamic and heat transfer performance of GE-E3 rotor blade squealer tip. A parameter sensitivity analysis using the Sobol Indice method is carried out to identify the key parameters for aerothermal performance of the squealer tip. Wherein, the inlet total pressure and the inlet flow angle are considered as flow condition uncertainty parameters and cavity depth is considered as geometrical uncertainty parameters. The uncertainty analysis results showed that under the influence of the uncertain geometry and operating conditions, the heat flux of squealer tip basically conforms to the normal distribution and the statistical mean value of it increased by 28.30% relative to the design value and the probability of it deviating from the design value by 10% is as high as 86.59%. The statistical average of the squealer tip film cooling effectiveness is reduced by 23.89% compared to the design value, and the probability of it deviating from the design value by 10% is as high as 85.08%. The result of sensitivity analysis reveals that the uncertainty of the aerodynamic characteristics of the squealer tip is almost entirely caused by the inlet flow angle which accounts for 94.01% of the variance of the leakage flow rate. And it is also the dominant variable for the uncertainty of the heat transfer performance considering that its variance indexes for tip heat flux QTip and film cooling effectiveness η are 67.71% and 54.42% respectively. Compared with the main effects, the influence of the interaction effects among the variables on the squealer tip aerothermal performance is almost negligible.

Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures.

Uncertainty quantification for complex deep learning models is increasingly important as these techniques see growing use in high-stakes, real-world settings. Currently, the quality of a model’s uncertainty is evaluated using point-prediction metrics, such as the negative log-likelihood (NLL), expected calibration error (ECE) or the Brier score on held-out data. Marginal coverage of prediction intervals or sets, a well-known concept in the statistical literature, is an intuitive alternative to these metrics but has yet to be systematically studied for many popular uncertainty quantification techniques for deep learning models. With marginal coverage and the complementary notion of the width of a prediction interval, downstream users of deployed machine learning models can better understand uncertainty quantification both on a global dataset level and on a per-sample basis. In this study, we provide the first large-scale evaluation of the empirical frequentist coverage properties of well-known uncertainty quantification techniques on a suite of regression and classification tasks. We find that, in general, some methods do achieve desirable coverage properties on in distribution samples, but that coverage is not maintained on out-of-distribution data. Our results demonstrate the failings of current uncertainty quantification techniques as dataset shift increases and reinforce coverage as an important metric in developing models for real-world applications.

Tackling Small Eigen-Gaps: Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise

This paper aims to address two fundamental challenges arising in eigenvector estimation and inference for a low-rank matrix from noisy observations: 1) how to estimate an unknown eigenvector when the eigen-gap (i.e. the spacing between the associated eigenvalue and the rest of the spectrum) is particularly small; 2) how to perform estimation and inference on linear functionals of an eigenvector—a sort of “fine-grained” statistical reasoning that goes far beyond the usual <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> analysis. We investigate how to address these challenges in a setting where the unknown <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n\times n$ </tex-math></inline-formula> matrix is symmetric and the additive noise matrix contains independent (and non-symmetric) entries. Based on eigen-decomposition of the asymmetric data matrix, we propose estimation and uncertainty quantification procedures for an unknown eigenvector, which further allow us to reason about linear functionals of an unknown eigenvector. The proposed procedures and the accompanying theory enjoy several important features: 1) distribution-free (i.e. prior knowledge about the noise distributions is not needed); 2) adaptive to heteroscedastic noise; 3) minimax optimal under Gaussian noise. Along the way, we establish valid procedures to construct confidence intervals for the unknown eigenvalues. All this is guaranteed even in the presence of a small eigen-gap (up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(\sqrt {n/\mathrm {poly}\log (n)}\,)$ </tex-math></inline-formula> times smaller than the requirement in prior theory), which goes significantly beyond what generic matrix perturbation theory has to offer.

Uncertainty due to discretization on the ALE algorithm for predicting water slamming loads

Numerical uncertainty due to discretization on the Arbitrary Lagrangian-Eulerian (ALE) Finite Element method is investigated in the study. The paper quantifies uncertainty using two ITTC recommended methods, and also applies a constant Courant-Friedrichs-Lewy (CFL) number based discretization approach, instead of performing the independent grid and time-based discretization recommended by ITTC. As a case study, water entry of a flat bottom rigid and flexible plate is simulated considering various entry velocities. The total slamming loads and structural responses on both the rigid and elastic bottom plates are predicted and validated against available experimental data. Results indicate that numerical errors due to discretization differ in the various parameters and from case to case. They do affect the analysis of slamming loads and associated structural responses, and the hydroelasticity analysis as well. The hydroelasticity effects on the slamming force generally increase as the entry velocity increases, however, the quantitative results differ much for models with different grids. For example, when the hydroelasticity effect is estimated using the finer model, the deviation of the total slamming force on the elastic plate relative to the one on the rigid body are 56%, 57%, and 63% respectively for the three constant entry velocities, whereas the estimations are −27%, −4% and 3% with the coarser model. The study concludes that the uncertainty due to discretization in ALE is not just case-specific, but also parameter specific. The uncertainty quantification procedures with a constant CFL number based refinement are recommended to investigate the uncertainty comparing to the individual grid and time step study, in particular for the ALE solution where the time step is adjusted automatically as the grid changes. Thus, consideration should be given to updating the ITTC guidelines to incorporate the constant CFL based discretization approach.

A Reproducing Kernel Hilbert Space Approach to Functional Calibration of Computer Models

This article develops a frequentist solution to the functional calibration problem, where the value of a calibration parameter in a computer model is allowed to vary with the value of control variables in the physical system. The need of functional calibration is motivated by engineering applications where using a constant calibration parameter results in a significant mismatch between outputs from the computer model and the physical experiment. Reproducing kernel Hilbert spaces (RKHS) are used to model the optimal calibration function, defined as the functional relationship between the calibration parameter and control variables that gives the best prediction. This optimal calibration function is estimated through penalized least squares with an RKHS-norm penalty and using physical data. An uncertainty quantification procedure is also developed for such estimates. Theoretical guarantees of the proposed method are provided in terms of prediction consistency and consitency of estimating the optimal calibration function. The proposed method is tested using both real and synthetic data and exhibits more robust performance in prediction and uncertainty quantification than the existing parametric functional calibration method and a state-of-art Bayesian method.

Data-driven inverse uncertainty quantification in the transverse tensile response of carbon fiber reinforced composites

Uncertainty quantification is critical for the full exploitation of composite materials’ potential. Inverse methods offer the possibility of indirectly characterizing the uncertainty of microscopic parameters by employing data sets from standard structural tests in higher scales. Two crucial requirements though, are the efficient modeling especially for the nonlinear prediction, and the measurement error availability from the tests which affects the updated scatter. This study employs effective stiffness and strength experimental data in order to quantify uncertainties of a carbon fiber UD composite in the microscale. A polynomial chaos surrogate model is trained from finite element simulations, able to efficiently predict the homogenized stiffness and strength for the uncertainty quantification procedure. The random parameters which are influential enough to be updated, are identified via a variance-based global sensitivity analysis. The inverse problem is solved with the Bayesian inference method, which updates any prior estimation of the probability models of the input parameters, based on output observations from the tests. Results show significant uncertainty reduction in comparison with typically used variance values in the literature and can be used to enrich the composite material databases. The proposed methodology is applied for the transverse tensile load case, although its non-intrusive nature allows applications for more load cases and various setups.

Editorial for the special issue on advances in forward and inverse surrogate modeling for high‐frequency design

Editorial for the special issue on advances in forward and inverse surrogate modeling for <scp>high‐frequency</scp> design

Enhancement of risk informed validation framework for external hazard scenario

In recent years, the U.S. Nuclear Regulatory Commission (USNRC) and the International Atomic Energy Agency (IAEA) have developed methodologies to assess the vulnerabilities of nuclear plants against site specific extreme hazards. In many cases, advanced simulation tools are being considered to simulate multi-physics, multi-scale phenomena and to evaluate vulnerability of nuclear facilities. The credibility of advanced simulation tools is assessed based on a formal verification, validation, and uncertainty quantification procedure. One of the key limitations in validation is the lack of relevant experimental data at system-level. This limitation leads to a decrease in the confidence of system-level risk predictions. Therefore, a robust validation framework is needed to formalize the confidence in predictive capability of advanced simulation results. This study enhances the existing risk informed validation methodology, originally proposed by Kwag et al. [1] and Bodda et al. [2], by developing additional attributes and a new set of validation indicies for a complete and wider applicability of the framework. In this manuscript, the methodology to identify the critical path that leads to the system-level failure is illustrated. The overall validation is checked for completeness and consistency by comparing the critical path for both the system-level simulation and experimental models. The applicability of the code for an intended application is represented in terms of various maturity levels and helps in the process of decision making.

Uncertainty Quantification in Vehicle Content Optimization for General Motors

A vehicle content portfolio refers to a complete set of combinations of vehicle features offered while satisfying certain restrictions for the vehicle model. Vehicle Content Optimization (VCO) is a simulation-based decision support system at General Motors (GM) that helps to optimize a vehicle content portfolio to improve GM’s business performance and customers’ satisfaction. VCO has been applied to most major vehicle models at GM. VCO consists of several steps that demand intensive computing power, thus requiring trade-offs between the estimation error of the simulated performance measures and the computation time. Given VCO’s substantial influence on GM’s content decisions, questions were raised regarding the business risk caused by uncertainty in the simulation results. This paper shows how we successfully established an uncertainty quantification procedure for VCO that can be applied to any vehicle model at GM. With this capability, GM can not only quantify the overall uncertainty in its performance measure estimates but also identify the largest source of uncertainty and reduce it by allocating more targeted simulation effort. Moreover, we identified several opportunities to improve the efficiency of VCO by reducing its computational overhead, some of which were adopted in the development of the next generation of VCO.

Improving Forecast Uncertainty Quantification by Incorporating Production History and Using a Modified Ranking Method of Geostatistical Realizations

Abstract The majority of the geostatistical realizations ranking methods disregard the production history in selection of realizations, due to its requirement of high simulation run time. They also ignore to consider the degree of linear relationship between the “ranks based on the ranking measure” and “ranks based on the performance parameter” in choosing the employed ranking measure. To address these concerns, we propose an uncertainty quantification workflow, which includes two sequential stages of history matching and realization selection. In the first stage, production data are incorporated in the uncertainty quantification procedure through a history matching process. A fast simulator is employed to find the realizations with consistent flow behavior with the production history data in shorter time, compared to a comprehensive simulator. The selected realizations are the input of the second stage of the workflow, which can be any type of the realization selection method. In this study, we used the most convenient realization selection method, i.e., ranking of the realizations. To select the most efficient ranking measure, we investigated the degree of the linear correlation between the ranks based on the several ranking measures and the performance parameter. In addition, due to the shortcomings of the traditional ranking methods in uncertainty quantiles identification, a modified ranking method is introduced. This modification increases the certainty in the probability of the selected realizations. The obtained results on 3D close-to-real synthetic reservoir models revealed the capability of the modified ranking method in more accurate quantification of the uncertainty in reservoir performance prediction.

Model Updating of Multiple Nominally Identical Car Components

A method for estimation of rubber bushing stiffness parameters is presented. Four individual rubber bushings, mounted in a car rear subframe are considered. A traditional model of the bushing elements using a generalised spring model, known as a CBUSH element in Nastran, is compared to a geometrically more realistic approach where the bushing is modelled with solid elements and a linear elastic material model. Each bushing is mass loaded to better reveal the bushing’s dynamic behaviour in a lower frequency range of interest. In an initial step, the overall subframe model is updated towards test data. In a second step, the bushing parameters are updated. Three nominally identical components are used to investigate the spread between the identified parameters. The model updating procedure is based on frequency responses and equalised damping. The undamped behaviour at frequencies below 300 Hz are considered. To quantify the parameter uncertainty, with respect to measurement noise for each individual, an uncertainty quantification procedure is proposed, using a linear-in-parameters surrogate model with bootstrapping.

Response analysis of an accelerating unbalanced rotating system with both random and interval variables

This paper investigates the accelerating up transient vibrations of a rotor system under both the random and uncertain-but-bounded parameters. The Polynomial Chaos Expansion (PCE) coupled with the Chebyshev Surrogate Method (CSM) is used to analyses the propagations of the two categorizes of uncertainties. The output responses will possess the characteristics of both bounded quantities and statistical moments. As a hybrid non-intrusive uncertainty quantification (UQ) procedure, the deterministic rotor model is taken as a black box and will only be executed at some parameter points. A number of uncertain physical parameters are studied and the corresponding transient responses are presented. The accuracy and efficiency are verified by the Monte Carlo simulations (MCS) in combination with the scanning scheme and also other hybrid analysis framework. It will provide guidance for the accurate transient dynamic analysis of engineering problems with hybrid uncertainties.

Impact of solar wind fluctuations on Electric Sail mission design

The Electric Solar Wind Sail (E-sail) is a propellantless propulsion system that generates thrust by exploiting the interaction between a grid of tethers, kept at a high electric potential, and the charged particles of the solar wind. Such an advanced propulsion system allows innovative and exotic mission scenarios to be envisaged, including non-Keplerian orbits, artificial Lagrange point maintenance, and heliostationary condition attainment. In the preliminary mission analysis of an E-sail-based spacecraft, the local physical properties of the solar wind are usually specified and kept constant, while the E-sail propulsive acceleration is assumed to vary with the heliocentric distance, the sail attitude, and the grid electric voltage. However, the solar wind physical properties are known to be characterized by a marked variability, which implies a non-negligible uncertainty as to whether or not the solutions obtained with a deterministic approach are representative of the actual E-sail trajectory. The aim of this paper is to propose an effective method to evaluate the impact of solar wind variability on the E-Sail trajectory design, by considering the solar wind dynamic pressure as a random variable with a gamma distribution. In particular, the effects of plasma property fluctuations on E-sail trajectory are calculated with an uncertainty quantification procedure based on the generalized polynomial chaos method. The paper also proposes a possible control strategy that uses suitable adjustments of grid electric voltage. Numerical simulations demonstrate the importance of such a control system for missions that require a precise modulation of the propulsive acceleration magnitude.

Towards an understanding of variations in the buckling of tailored variable angle tow composite plates

In this paper, variable angle tow (VAT) composite plates tailored to enhance buckling performance are studied with the use of stochastic finite element method to quantify uncertainties in buckling measures arising from variations in material properties and fibre tow path. Detailed formulations for predicting buckling statistics in terms of mean value and standard deviation are derived to enable a perturbation-based stochastic finite element analysis. The derivations are built on a linear variation formula for fibre tow path and plate element based on the first order shear deformation theory. They are integrated with Taylor series expansion to propagate uncertainties from inputs to buckling performance measures, including buckling eigenvalues, critical buckling coefficients, etc. A twelve-layer VAT composite plate, with optimally designed fibre tow paths under various boundary conditions, has been investigated to illustrate the uncertainty quantification procedure. The performance of the perturbation-based stochastic finite element method has been validated using Monte Carlo simulation. Influences of variations in material properties and fibre tow path are thoroughly examined to understand the variability of buckling performance of VAT composites.

A Stochastic Model Predictive Control Framework for Stationary Battery Systems

A stochastic model predictive control (MPC) framework is presented to determine real-time commitments in energy and frequency regulation markets for a stationary battery while simultaneously mitigating long-term demand charges for an attached load. The control problem is multi-scale in nature and poses challenges on computational tractability of the stochastic program and of forecasting and uncertainty quantification (UQ) procedures. The framework deals with tractability of the stochastic program by using a discounting factor for long-term demand charges, while a Ledoit–Wolf covariance estimator is used to overcome UQ tractability issues. The performance of stochastic MPC is benchmarked against that of perfect information MPC and deterministic MPC for different prediction horizon lengths and demand charge discounting strategies. A case study using real load data for a typical university campus and price and regulation data from PJM is considered. It is found that stochastic MPC can recover 83% of the ideal value of the battery, which is defined as the expected savings obtained by operating the battery under perfect information MPC. In contrast, deterministic MPC can only recover 73% of this ideal value. It is also found that operating the battery under stochastic MPC improves the battery payback period by 12.1%, while operating it under perfect information improves it by 27.9%.