Uncertainty Quantification Methods Research Articles

A reduced order/simplified method for uncertainty quantification on the floating wind turbine design basis

The cost of floating wind turbines (FWTs) is currently higher than onshore and bottom fixed offshore wind turbines. An important step towards more cost-efficient FWT structures is understanding the uncertainties and how they affect the design performance. On the path towards developing a fully probabilistic design approach, an important initial step is defining the variables and uncertainties that significantly impact the final design performance. This study aims to conduct a design sensitivity analysis on a FWT system consisting of the VolturnUS Semisubmersible floater coupled with an IEA 15 MW turbine. First, a Python-based design evaluation framework is implemented that computes operational limit states as a function of design input parameters defining the floater design. The system responses and the loads are obtained using Horizontal Axis Wind turbine simulation Code 2nd generation (HAWC2). Time domain outputs of the design evaluation framework are compared to a frequency domain response using Quick Load Analysis of Floating Wind Turbines (QuLAF). Initial results provide that buoyancy column diameter and the floater radius have the highest effects on the general floater response.

Entropy-Conservative Discontinuous Galerkin Methods for the Shallow Water Equations with Uncertainty

In this paper, we develop an entropy-conservative discontinuous Galerkin (DG) method for the shallow water (SW) equation with random inputs. One of the most popular methods for uncertainty quantification is the generalized Polynomial Chaos (gPC) approach which we consider in the following manuscript. We apply the stochastic Galerkin (SG) method to the stochastic SW equations. Using the SG approach in the stochastic hyperbolic SW system yields a purely deterministic system that is not necessarily hyperbolic anymore. The lack of the hyperbolicity leads to ill-posedness and stability issues in numerical simulations. By transforming the system using Roe variables, the hyperbolicity can be ensured and an entropy-entropy flux pair is known from a recent investigation by Gerster and Herty (Commun. Comput. Phys. 27(3): 639–671, 2020). We use this pair and determine a corresponding entropy flux potential. Then, we construct entropy conservative numerical two-point fluxes for this augmented system. By applying these new numerical fluxes in a nodal DG spectral element method (DGSEM) with flux differencing ansatz, we obtain a provable entropy conservative (dissipative) scheme. In numerical experiments, we validate our theoretical findings.

Efficient and accurate uncertainty quantification in engineering simulations using time-separated stochastic mechanics

A robust method for uncertainty quantification is undeniably leading to a greater certainty in simulation results and more sustainable designs. The inherent uncertainties of the world around us render everything stochastic, from material parameters, over geometries, up to forces. Consequently, the results of engineering simulations should reflect this randomness. Many methods have been developed for uncertainty quantification for linear elastic material behavior. However, real-life structure often exhibit inelastic material behavior such as visco-plasticity. Inelastic material behavior is described by additional internal variables with accompanying differential equations. This increases the complexity for the computation of stochastic quantities, e.g., expectation and standard deviation, drastically. The time-separated stochastic mechanics is a novel method for the uncertainty quantification of inelastic materials. It is based on a separation of all fields into a sum of products of time-dependent but deterministic and stochastic but time-independent terms. Only a low number of deterministic finite element simulations are then required to track the effect of (in)homogeneous material fluctuations on stress and internal variables. Despite the low computational effort the results are often indistinguishable from reference Monte Carlo simulations for a variety of boundary conditions and loading scenarios.

A U-Net architecture as a surrogate model combined with a geostatistical spectral algorithm for transient groundwater flow inverse problems

Characterizing groundwater flow parameters is crucial for understanding complex aquifer systems, and inverse techniques play a fundamental role in modeling hydrogeological parameters and assessing their uncertainties. Nonetheless, the use of a forward model in these methods can be highly time-consuming, especially with an increasing number of model parameters. To address this issue, we propose a surrogate model based on a U-Net architecture that replaces the transient groundwater flow model, reducing runtime and enabling a fast quantification of uncertainties related to key parameters, including heterogeneous hydraulic conductivity, boundary conditions, specific storage, and pumping rate. The surrogate is trained using limited evaluations of the forward model to learn the physical relationship between hydraulic conductivity fields and transient hydraulic heads measured on-site. The physical principles of the studied problem, including boundary conditions, specific storage, and source terms, are also mapped and introduced as inputs to the model to enhance its understanding of the governing equation of transient groundwater flow. To speed up learning using image–image regression, the previously predicted transient hydraulic heads also serve as an input to predict the transient heads at the current time step. Once the model is trained, we use a spectral geostatistical method to solve the inverse problem, a pumping test of 12 h, using the surrogate model in place of the forward model. Our study demonstrates that the trained U-Net accurately reproduces the state variables corresponding to a specific parameter field, and in terms of computational demand, using U-Net as a surrogate model reduces the required computational time by approximately an order of magnitude for the defined problem. The proposed approach offers an efficient and accurate method for groundwater flow parameter characterization and uncertainty quantification in complex aquifer systems.

IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning

We propose a novel framework for uncertainty quantification via information bottleneck (IB-UQ) for scientific machine learning tasks, including deep neural network (DNN) regression and neural operator learning (DeepONet). Specifically, we incorporate the bottleneck by a confidence-aware encoder, which encodes inputs into latent representations according to the confidence of the input data belonging to the region where training data is located, and utilize a Gaussian decoder to predict means and variances of outputs conditional on representation variables. Furthermore, we propose a data augmentation based information bottleneck objective which can enhance the quantification quality of the extrapolation uncertainty, and the encoder and decoder can be both trained by minimizing a tractable variational bound of the objective. In comparison to uncertainty quantification (UQ) methods for scientific learning tasks that rely on Bayesian neural networks with Hamiltonian Monte Carlo posterior estimators, the model we propose is computationally efficient, particularly when dealing with large-scale datasets. The effectiveness of the IB-UQ model has been demonstrated through several representative examples, such as regression for discontinuous functions, learning nonlinear operators for partial differential equations, and a large-scale climate map model. The experimental results indicate that the IB-UQ model can handle noisy data, generate robust predictions, and provide confident uncertainty evaluation for out-of-distribution data.

Effects of compositional uncertainties in cracked NH3/biosyngas fuel blends on the combustion characteristics and performance of a combined-cycle gas turbine: A numerical thermokinetic study

Blending of partially cracked ammonia with biosyngas is an attractive strategy for improving NH3 combustion. In practice, products of biomass gasification and those of thermo-catalytic cracking of NH3 are subject to some compositional uncertainties. Despite their practical importance, so far, the effects of such uncertainties on combustion systems remained largely unexplored. Hence, this paper quantifies the effects of small compositional uncertainties of reactants upon combustion of partially cracked NH3/syngas/air mixtures. An uncertainty quantification method, based on polynomial chaos expansion and a data-driven model, is utilised to investigate the effects of uncertainty in fuel composition on the laminar flame speed (SL) and adiabatic flame temperature (Tad) at different inlet pressures (Pi). The analysis is then extended to the power output of a combined-cycle gas turbine fuelled by the reactants. It is found that 1.5% fuel compositional uncertainty can cause 12–21% of SL uncertainty depending on the inlet pressure. Furthermore, the effect of compositional uncertainty on Tad increases at higher ratios of H2 to NH3. Sensitivity analysis reveals that the uncertainty of CO contribution to SL uncertainty is higher than that of NH3, while the trend is reversed for the Tad uncertainty. In addition, the power output from the combined-cycle gas turbine system varies between 4 and 6% with 1.5% of fuel compositional uncertainty. This become more noticeable at elevated Pi [5–10 atm], particularly when the fuel mixture contains high H2 which is the main contributor to Tad variability.

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

Physics-based reduced order modeling for uncertainty quantification of guided wave propagation using Bayesian optimization

Guided wave propagation (GWP) is commonly employed for the design of SHM systems. However, GWP is sensitive to variations in the material properties, often leading to false alarms. To address this issue, uncertainty quantification (UQ) is employed to improve the reliability of the predictions. Computational structural mechanics is a useful tool for the simulation of GWP. Even so, the application of UQ methods requires numerous solutions, while large-scale, transient GWP simulations increase the computational cost. In this paper, we propose a machine learning (ML)-based reduced order model (ROM), annotated as BO-ML-ROM, to decrease the computational time of the GWP simulation. The ROM is integrated with a Bayesian optimization (BO) framework to adaptively sample the data for the ROM training. The results showed that BO outperforms one-shot sampling methods, both in terms of accuracy and speed. The developed ROM, which is orders of magnitude faster than the original solver, is then applied for forward uncertainty quantification of the GWP in an aluminum plate and a composite panel under varying material properties. The UQ analysis reveals the higher impact of the uncertainties in the wave reflections. Furthermore, to compute the influence of the material properties on the predictions, a sensitivity analysis (SA) is performed using the variance-based Sobol’ indices and the Shapley additive explanations, where it is shown that the uncertainties in the elastic properties have a varying influence across time. The predicted results reveal the efficiency of BO-ML-ROM for the simulation of the GWP and demonstrate its utility for UQ and SA.

Advanced Uncertainty Quantification for Flood Inundation Modelling

Flood hazards present a significant risk to the UK, with homes, businesses and critical infrastructure exposed to a mixture of fluvial, surface water and coastal flooding. Climate change is expected to influence river flows, changing the frequency and magnitude of future flood events. Flood hazard assessments are used by decision-makers to implement policies and engineering interventions to reduce the impacts of these flood events. Probabilistic flood modelling can explore input and parameter uncertainties in flood models to fully quantify inundation uncertainty. However, probabilistic methods require large computational costs—limiting their application. This paper investigates a range of advanced uncertainty quantification methods (traditional Monte Carlo (FMC), Kriging and multi-fidelity Monte Carlo (MFMC)) to reduce the dichotomy between accuracy and costs. Results suggest that Kriging can reduce computational costs by 99.9% over FMC. The significantly increased efficiency has the potential to improve future policy and engineering decisions, reducing the impacts of future flood events.

Analysis of machine learning prediction reliability based on sampling distance evaluation with feature decorrelation

Despite successful use in a wide variety of disciplines for data analysis and prediction, machine learning (ML) methods suffer from a lack of understanding of the reliability of predictions due to the lack of transparency and black-box nature of ML models. In materials science and other fields, typical ML model results include a significant number of low-quality predictions. This problem is known to be particularly acute for target systems which differ significantly from the data used for ML model training. However, to date, a general method for uncertainty quantification (UQ) of ML predictions has not been available. Focusing on the intuitive and computationally efficient similarity-based UQ, we show that a simple metric based on Euclidean feature space distance and sampling density together with the decorrelation of the features using Gram–Schmidt orthogonalization allows effective separation of the accurately predicted data points from data points with poor prediction accuracy. To demonstrate the generality of the method, we apply it to support vector regression models for various small data sets in materials science and other fields. We also show that this metric is a more effective UQ tool than the standard approach of using the average distance of k nearest neighbors (k = 1–10) in features space for similarity evaluation. Our method is computationally simple, can be used with any ML learning method and enables analysis of the sources of the ML prediction errors. Therefore, it is suitable for use as a standard technique for the estimation of ML prediction reliability for small data sets and as a tool for data set design.

Global ranking of the sensitivity of interaction potential contributions within classical molecular dynamics force fields

Uncertainty quantification (UQ) is rapidly becoming a sine qua non for all forms of computational science out of which actionable outcomes are anticipated. Much of the microscopic world of atoms and molecules has remained immune to these developments but due to the fundamental problems of reproducibility and reliability, it is essential that practitioners pay attention to the issues concerned. Here a UQ study is undertaken of classical molecular dynamics with a particular focus on uncertainties in the high-dimensional force-field parameters, which affect key quantities of interest, including material properties and binding free energy predictions in drug discovery and personalized medicine. Using scalable UQ methods based on active subspaces that invoke machine learning and Gaussian processes, the sensitivity of the input parameters is ranked. Our analyses reveal that the prediction uncertainty is dominated by a small number of the hundreds of interaction potential parameters within the force fields employed. This ranking highlights what forms of interaction control the prediction uncertainty and enables systematic improvements to be made in future optimizations of such parameters.

Uncertainty Benchmarks for Time-Dependent Transport Problems

Verification solutions for uncertainty quantification (UQ) are presented for time-dependent transport problems where , the scattering ratio, is uncertain. The method of polynomial chaos expansions is employed for quick and accurate calculation of the quantities of interest (QoIs), and uncollided solutions are used to treat part of the uncertainty calculation analytically. We find that approximately six moments in the polynomial expansion are required to represent the solutions to these problems accurately. Additionally, the results show that if the uncertainty interval spans c = 1, which means it is uncertain whether the system is multiplying or not, the confidence interval will grow in time. Finally, since the QoI is a strictly increasing function, the percentile values are known and can be used to verify the accuracy of the expansion. These results can be used to test UQ methods for time-dependent transport problems.

Constructing prediction intervals to explore uncertainty based on deep neural networks

The conventional approaches to constructing Prediction Intervals (PIs) always follow the principle of ‘high coverage and narrow width’. However, the deviation information has been largely neglected, making the PIs unsatisfactory. For high-risk forecasting tasks, the cost of forecast failure may be prohibitive. To address this, this work introduces a multi-objective loss function that includes Prediction Interval Accumulation Deviation (PIAD) within the Lower Upper Bound Estimation (LUBE) framework. The proposed model can achieve the goal of ‘high coverage, narrow width, and small bias’ in PIs, thus minimizing costs even in cases of prediction failure. A salient feature of the LUBE framework is its ability to discern uncertainty without explicit uncertainty labels, where the data uncertainty and model uncertainty are learned by Deep Neural Networks (DNN) and a model ensemble, respectively. The validity of the proposed method is demonstrated through its application to the prediction of carbon prices in China. Compared with conventional uncertainty quantification methods, the improved interval optimization method can achieve narrower PI widths.

AI-driven non-intrusive uncertainty quantification of advanced nuclear fuels for digital twin-enabling technology

In response to the urgent need to establish AI/ML-integrated Digital Twin (DT) technology within next-generation nuclear systems, advancements in modeling methods and simulation codes are necessary. The increased complexity of models demands significant computational resources to quantify their uncertainties. To address this challenge, a data-driven non-intrusive uncertainty quantification method via polynomial chaos expansion is introduced as an efficient strategy within the finite element analysis-based fuel performance code BISON. Models of UO2 and U3Si2 fuels, alongside SiC/SiC cladding material, were prepared to demonstrate the proposed method. The impact of four independent uncertain input variables on the system output was quantified, requiring fewer than 100 BISON simulations for each model. This approach not only accelerates the modeling and simulation task but also enhances the reliability in the development of DT-enabling technologies.

Design of polycrystalline metallic alloys under multi-scale uncertainty by connecting atomistic to meso‑scale properties

Uncertainty quantification plays a pivotal role in advancing the development of reliable and high-performance material designs via multiscale materials modeling. This study focuses on modeling the uncertainty of meso‑scale mechanical properties, e.g. modulus of elasticity and yield strength of Ti-7Al, by incorporating the inherent randomness of substitutional atoms of the alloy and its microstructural texture. A molecular dynamics approach is employed to probe the epistemic uncertainty of single-crystal properties. Moreover, the aleatoric uncertainty is quantified using the experimental microstructural texture data of a previous investigation. The propagation of the epistemic and aleatoric uncertainty on the homogenized properties is investigated using an analytical uncertainty quantification method. Subsequently, the Ti-7Al alloy microstructures are designed to improve meso‑scale mechanical properties under uncertainty by integrating the analytical method into the optimization scheme.

Reliability assessment of guided wave damage localization with deep learning uncertainty quantification methods

Guided wave-based structural health monitoring is an attractive option for detecting structural defects in an automated manner. In this work, we focus on the task of damage localization. Deep learning methods have been shown to have superior performance for damage localization. Yet, environmental variations introduce uncertainty in the system and reduce its reliability. For this reason, it is crucial to assess the reliability of estimates taken from structural health monitoring systems. In this work, we estimate the localization reliability from a single snapshot of sparse array guided wave measurements instead of reporting values averaged over an entire set of test measurements. The assessment strategy can be added to any deep learning localization model and produces both a localization and uncertainty estimate. The deep learning model is trained using only guided wave simulations. We assess the uncertainty using both simulated and experimental data with temperature variations. Multiple deep learning-based uncertainty quantification methods are analyzed. Results demonstrate correlations between uncertainty, temperature variations, and the presence of synthetic damage. We also compare with reliability derived from delay-and-sum localization. We find that a deep ensemble learning strategy provides the most reliable damage localization and uncertainty quantification.

Robust Uncertainty Quantification Using Conformalised Monte Carlo Prediction

Deploying deep learning models in safety-critical applications remains a very challenging task, mandating the provision of assurances for the dependable operation of these models. Uncertainty quantification (UQ) methods estimate the model’s confidence per prediction, informing decision-making by considering the effect of randomness and model misspecification. Despite the advances of state-of-the-art UQ methods, they are computationally expensive or produce conservative prediction sets/intervals. We introduce MC-CP, a novel hybrid UQ method that combines a new adaptive Monte Carlo (MC) dropout method with conformal prediction (CP). MC-CP adaptively modulates the traditional MC dropout at runtime to save memory and computation resources, enabling predictions to be consumed by CP, yielding robust prediction sets/intervals. Throughout comprehensive experiments, we show that MC-CP delivers significant improvements over comparable UQ methods, like MC dropout, RAPS and CQR, both in classification and regression benchmarks. MC-CP can be easily added to existing models, making its deployment simple. The MC-CP code and replication package is available at https://github.com/team-daniel/MC-CP.

Multifidelity methods for uncertainty quantification of a nonlocal model for phase changes in materials

This study is devoted to the construction of a multifidelity Monte Carlo (MFMC) method for the uncertainty quantification of a nonlocal, non-mass-conserving Cahn-Hilliard model for phase transitions with an obstacle potential. We are interested in estimating the expected value of an output of interest (OoI) that depends on the solution of the nonlocal Cahn-Hilliard model. As opposed to its local counterpart, the nonlocal model captures sharp interfaces without the need for significant mesh refinement. However, the computational cost of the nonlocal Cahn-Hilliard model is higher than that of its local counterpart with similar mesh refinement, inhibiting its use for outer-loop applications such as uncertainty quantification. The MFMC method augments the desired high-fidelity, high-cost OoI with a set of lower-fidelity, lower-cost OoIs to alleviate the computational burden associated with nonlocality. Most of the computational budget is allocated to sampling the cheap surrogate models to achieve speedup, whereas the high-fidelity model is sparsely sampled to maintain accuracy. For the non-mass-conserving nonlocal Cahn-Hilliard model, the use of the MFMC method results in, for a given computational budget, about an order of magnitude reduction in the mean-squared error of the expected value of the OoI relative to that of the Monte Carlo method.

On the uncertainty of long-period return values of extreme daily precipitation

Methods for calculating return values of extreme precipitation and their uncertainty are compared using daily precipitation rates over the Western U.S. and Southwestern Canada from a large ensemble of climate model simulations. The roles of return-value estimation procedures and sample size in uncertainty are evaluated for various return periods. We compare two different generalized extreme value (GEV) parameter estimation techniques, namely L-moments and maximum likelihood (MLE), as well as empirical techniques. Even for very large datasets, confidence intervals calculated using GEV techniques are narrower than those calculated using empirical methods. Furthermore, the more efficient L-moments parameter estimation techniques result in narrower confidence intervals than MLE parameter estimation techniques at small sample sizes, but similar best estimates. It should be noted that we do not claim that either parameter fitting technique is better calibrated than the other to estimate long period return values. While a non-stationary MLE methodology is readily available to estimate GEV parameters, it is not for the L-moments method. Comparison of uncertainty quantification methods are found to yield significantly different estimates for small sample sizes but converge to similar results as sample size increases. Finally, practical recommendations about the length and size of climate model ensemble simulations and the choice of statistical methods to robustly estimate long period return values of extreme daily precipitation statistics and quantify their uncertainty.

A fast uncertainty quantification methodology and sampling technique for joint probability distribution of the Arrhenius rate expression: a case study applied to H2/CO kinetic mechanism

This work proposes a fast, novel, mathematically robust, and elegant unconstrained Method of Uncertainty Quantification (MUQ) for the temperature-dependent Arrhenius rate constant using Cholesky Decomposition (CD) of the covariance matrix of the Arrhenius parameters. The Arrhenius parameters of a reaction are treated as normally distributed correlated random variables. The MUQ method lends itself to an approach for Sampling of Arrhenius Curves (SAC), which automatically ensures that the generated samples are consistent with the distribution of Arrhenius parameters. The Method of Uncertainty Quantification and Sampling of Arrhenius Curves (MUQ-SAC) is used to train a quadratic polynomial response surface (PRS). Three important classes of Arrhenius curves possible within the SAC method are identified. From the total set of targets, a small subset of targets is first used to study the combinations of different classes of Arrhenius curves and their proportions within the design matrix, which is used to train PRS. Based on this understanding, response surfaces are generated for 64 ignition delay targets (Tig), which comprises of a wide range of pressure (P: 1--32.7 atm), temperature (T: 916-DIFadd-2869 K), and equivalence ratio (ϕ: 0.5--6.11), as well as for 74 laminar flame speed targets (Fls), spanning P: 1--25 atm, T: 285--600 K, and ϕ: 0.6--5. The H 2 /CO sub-mechanism of FFCM1.0 is chosen for the case study in which 22 reactions (66 Arrhenius parameters) are considered active parameters. The quality of the generated PRS is measured using the relative Maximum Residual Error (MRE). The quality of PRS is also checked against the search iterations encountered during mechanism optimisation. Accurate PRS could be generated over the search iterations, validating that the SAC method is able to span the entire uncertainty domain while training the PRS. The implementation of the MUQ-SAC method is made available at https://github.com/KrunalPanchal1995/MUQ-SAC.git.