Comprehensive Uncertainty Quantification Research Articles

Very small‐scale, segregating‐fluidized‐bed experiments: A dataset for CFD‐DEM validation and uncertainty quantification

AbstractFluidization experiments were conducted on a small scale and with a rapid response (short duration) to enable corresponding simulations at low‐computational cost. Rise times are reported for four or fewer polyethylene particles (intruders) in an air‐fluidized bed of ~5000 group D glass beads. Experimental inputs were completely characterized—particle properties, system dimensions and operating conditions—which is necessary for validating computational fluid mechanics (CFD)‐discrete element method (DEM) including a comprehensive uncertainty quantification (UQ) analysis. Input uncertainties are reported as bounds or cumulative distribution functions of measured values. The staggering number of simulations required to complete a UQ analysis (~O[104] simulations corresponding to ~5 uncertain inputs) motivates this study. These segregating‐bed experiments are designed to permit analogous CFD‐DEM simulations to complete in less than a day on a single (~2.5 GHz) computational processor unit (CPU). Segregation times are reported for several operating conditions, intruder sizes, and initial configurations, providing a rich dataset for numerical model testing, validation and UQ.

Flow field tomography with uncertainty quantification using a Bayesian physics-informed neural network

Abstract We report a new approach to flow field tomography that uses the Navier–Stokes and advection–diffusion equations to regularize reconstructions. Tomography is increasingly employed to infer 2D or 3D fluid flow and combustion structures from a series of line-of-sight (LoS) integrated measurements using a wide array of imaging modalities. The high-dimensional flow field is reconstructed from low-dimensional measurements by inverting a projection model that comprises path integrals along each LoS through the region of interest. Regularization techniques are needed to obtain realistic estimates, but current methods rely on a penalty term that is incompatible with the flow physics to varying degrees. Physics-informed neural networks (PINNs) provide a new framework for inverse analysis that enables regularization of the flow field estimates using the governing physics. We demonstrate how a PINN can be leveraged to reconstruct a 2D flow field from sparse LoS-integrated measurements with no knowledge of the boundary conditions by incorporating the measurement model into the loss function used to train the network. The resulting reconstructions are remarkably superior to reconstructions produced by state-of-the-art algorithms, even when a PINN is used for post-processing. However, as with conventional iterative algorithms, our approach is susceptible to semi-convergence when there is a high level of noise. We address this issue through the use of a Bayesian PINN, which facilitates comprehensive uncertainty quantification of the reconstructions, enables the use of a more intuitive loss function, and reveals the source of semi-convergence.

Towards the NASA UQ Challenge 2019: Systematically forward and inverse approaches for uncertainty propagation and quantification

This paper is dedicated to exploring the NASA Langley Challenge on Optimization under Uncertainty by proposing a series of approaches for both forward and inverse treatment of uncertainty propagation and quantification. The primary effort is placed on the categorization of the subproblems as to be forward or inverse procedures, such that dedicated techniques are proposed for the two directions, respectively. The sensitivity analysis and reliability analysis are categorized as forward procedures, while modal calibration & uncertainty reduction, reliability-based optimization, and risk-based design are regarded as inverse procedures. For both directions, the overall approach is based on imprecise probability characterization where both aleatory and epistemic uncertainties are investigated for the inputs, and consequently, the output is described as the probability-box (P-box). Theoretic development is focused on the definition of comprehensive uncertainty quantification criteria from limited and irregular time-domain observations to extract as much as possible uncertainty information, which will be significant for the inverse procedure to refine uncertainty models. Furthermore, a decoupling approach is proposed to investigate the P-box along two directions such that the epistemic and aleatory uncertainties are decoupled, and thus a two-loop procedure is designed to propagate both epistemic and aleatory uncertainties through the systematic model. The key for successfully addressing this challenge is in obtaining on the balance among an appropriate hypothesis of the input uncertainty model, a comprehensive criterion of output uncertainty quantification, and a computational viable approach for both forward and inverse uncertainty treatment.

Gridded daily weather data for North America with comprehensive uncertainty quantification

Access to daily high-resolution gridded surface weather data based on direct observations and over long time periods is essential for many studies and applications including vegetation, wildlife, soil health, hydrological modelling, and as driver data in Earth system models. We present Daymet V4, a 40-year daily meteorological dataset on a 1 km grid for North America, Hawaii, and Puerto Rico, providing temperature, precipitation, shortwave radiation, vapor pressure, snow water equivalent, and day length. The dataset includes an objective quantification of uncertainty based on strict cross-validation analysis for temperature and precipitation results. The dataset represents several improvements from a previous version, and this data descriptor provides complete documentation for updated methods. Improvements include: reductions in the timing bias of input reporting weather station measurements; improvement to the three-dimensional regression model techniques in the core algorithm; and a novel approach to handling high elevation temperature measurement biases. We show cross-validation analyses with the underlying weather station data to demonstrate the technical validity of new dataset generation methods, and to quantify improved accuracy.

A comprehensive uncertainty quantification of large-scale process-based crop modeling frameworks

Regional and global impact assessment tools are increasingly used to explore and evaluate the impact of climate change and extreme events on crop yield and environmental externalities. However, the large uncertainties associated with the inputs or the parameters in crop models within these tools, limits their predictive ability, exceeding the spatiotemporal variability of observed yields. The objective of this study is to explore and quantify different sources of uncertainties and assumptions made behind initial conditions (IC), soil input, meteorological forcing, management practices and model cultivar parameters by running regional simulations for the time period between 2009 and 2019. Simulations were performed for maize and soybean using the pSIMS platform across the U.S Midwest by incrementally accounting for five sources of uncertainty with a resolution using the APSIM and DSSAT crop growth models. First, the relative contribution of different sources of uncertainty was estimated over time and space. Then, a series of nitrate leaching hotpots were identified and a regional maize yield productivity index was estimated by decomposing the uncertainty in the same scenario using a hierarchical Bayesian random-effect model. All factors showed a strong spatial pattern in their contribution to the total uncertainty and their contribution was found to be partially dependent on location. However, across the whole region, it was found that the uncertainty around management is larger than IC, soil and meteorological forcing while showing a strong correlation with each of these factors. Given the high spatial correlation, we hypothesize that constraining soil inputs and management uncertainty could allow for the largest reduction in predictive uncertainty for crop yield. Our results showed vast areas over northern IA, IL and IN with high potential for NO3 leaching and southern IA, IL and east NE with lower maize productivity index compared to the regional average.

Multi-instrument Bayesian reconstruction of plasma shape evolution in the C-2W experiment

We determined the time-dependent geometry, including high-frequency oscillations, of the plasma density in TAE's C-2W experiment [Gota et al., Nucl. Fusion 59, 112009 (2019)]. This was done as a joint Bayesian reconstruction from a 14-chord FIR interferometer in the midplane, 32 Mirnov probes at the periphery, and 8 shine-through detectors at the targets of the neutral beams. For each point in time, we recovered, with credibility intervals, the radial density profile of the plasma; bulk plasma displacement; and amplitudes, frequencies, and phases of the azimuthal modes n = 1, …, 4. Also reconstructed were the radial profiles of the deformations associated with each of the azimuthal modes. Bayesian posterior sampling was done via Hamiltonian Monte Carlo with custom preconditioning. This gave us a comprehensive uncertainty quantification of the reconstructed values, including correlations and some understanding of multimodal posteriors.

Optimization or Bayesian Strategy? Performance of the Bhattacharyya Distance in Different Algorithms of Stochastic Model Updating

Abstract The Bhattacharyya distance has been developed as a comprehensive uncertainty quantification metric by capturing multiple uncertainty sources from both numerical predictions and experimental measurements. This work pursues a further investigation of the performance of the Bhattacharyya distance in different methodologies for stochastic model updating, and thus to prove the universality of the Bhattacharyya distance in various currently popular updating procedures. The first procedure is the Bayesian model updating where the Bhattacharyya distance is utilized to define an approximate likelihood function and the transitional Markov chain Monte Carlo algorithm is employed to obtain the posterior distribution of the parameters. In the second updating procedure, the Bhattacharyya distance is utilized to construct the objective function of an optimization problem. The objective function is defined as the Bhattacharyya distance between the samples of numerical prediction and the samples of the target data. The comparison study is performed on a four degrees-of-freedom mass-spring system. A challenging task is raised in this example by assigning different distributions to the parameters with imprecise distribution coefficients. This requires the stochastic updating procedure to calibrate not the parameters themselves, but their distribution properties. The second example employs the GARTEUR SM-AG19 benchmark structure to demonstrate the feasibility of the Bhattacharyya distance in the presence of practical experiment uncertainty raising from measuring techniques, equipment, and subjective randomness. The results demonstrate the Bhattacharyya distance as a comprehensive and universal uncertainty quantification metric in stochastic model updating.

Uncertainty Quantification in Sunspot Counts

Abstract Observing and counting sunspots constitutes one of the longest-running scientific experiments, with first observations dating back to Galileo (around 1610). Today the sunspot number (SN) time series acts as a benchmark of solar activity in a large range of physical models. An appropriate statistical modeling, adapted to the time series’ complex nature, is, however, still lacking. In this work, we provide the first comprehensive uncertainty quantification analysis of sunspot counts. We study three components: the number of sunspots (N s ), the number of sunspot groups (N g ), and the composite N c , defined as . Those are reported by a network of observatories around the world and are corrupted by errors of various types. We use a multiplicative framework to provide, for these three components, an estimation of their error distribution in various regimes (short-term, long-term, minima of solar activity). We also propose a robust estimator for the underlying solar signal and fit density distributions that take into account intrinsic characteristics such as overdispersion, excess of zeros, and multiple modes. The estimation of the solar signal underlying the composite N c may be seen as a robust version of the International Sunspot Number (ISN), widely used as a proxy of solar activity. Therefore, our results on N c may help characterize the uncertainty on ISN as well. Our results pave the way for a future monitoring of the observatories in quasi-real time, with the aim of alerting the observers when they start deviating from the network and preventing large drifts from occurring.

Comprehensive Uncertainty Quantification and Sensitivity Analysis for Cardiac Action Potential Models.

Recent efforts to ensure the reliability of computational model-based predictions in healthcare, such as the ASME V&V40 Standard, emphasize the importance of uncertainty quantification (UQ) and sensitivity analysis (SA) when evaluating computational models. UQ involves empirically determining the uncertainty in model inputs—typically resulting from natural variability or measurement error—and then calculating the resultant uncertainty in model outputs. SA involves calculating how uncertainty in model outputs can be apportioned to input uncertainty. Rigorous comprehensive UQ/SA provides confidence that model-based decisions are robust to underlying uncertainties. However, comprehensive UQ/SA is not currently feasible for whole heart models, due to numerous factors including model complexity and difficulty in measuring variability in the many parameters. Here, we present a significant step to developing a framework to overcome these limitations. We: (i) developed a novel action potential (AP) model of moderate complexity (six currents, seven variables, 36 parameters); (ii) prescribed input variability for all parameters (not empirically derived); (iii) used a single “hyper-parameter” to study increasing levels of parameter uncertainty; (iv) performed UQ and SA for a range of model-derived quantities with physiological relevance; and (v) present quantitative and qualitative ways to analyze different behaviors that occur under parameter uncertainty, including “model failure”. This is the first time uncertainty in every parameter (including conductances, steady-state parameters, and time constant parameters) of every ionic current in a cardiac model has been studied. This approach allowed us to demonstrate that, for this model, the simulated AP is fully robust to low levels of parameter uncertainty — to our knowledge the first time this has been shown of any cardiac model. A range of dynamics was observed at larger parameter uncertainty (e.g., oscillatory dynamics); analysis revealed that five parameters were highly influential in these dynamics. Overall, we demonstrate feasibility of performing comprehensive UQ/SA for cardiac cell models and demonstrate how to assess robustness and overcome model failure when performing cardiac UQ analyses. The approach presented here represents an important and significant step toward the development of model-based clinical tools which are demonstrably robust to all underlying uncertainties and therefore more reliable in safety-critical decision-making.

Uncertainty Quantification in Metallic Additive Manufacturing Through Physics-Informed Data-Driven Modeling

The complicated metal-based additive manufacturing (AM) process involves various sources of uncertainty, leading to variability in AM products. For comprehensive uncertainty quantification (UQ) of AM processes, we present a physics-informed data-driven modeling framework, in which multilevel data-driven surrogate models are constructed based on extensive computational data obtained by multiscale multiphysics AM models. It starts with computationally inexpensive surrogate models for which the uncertainty can be readily quantified, followed by global sensitivity analysis for comprehensive UQ study. Using AM-fabricated Ti-6Al-4V components as examples, this study demonstrates the capability of the proposed data-driven UQ framework for efficient investigation of uncertainty propagation from process parameters to material microstructures, then to macrolevel mechanical properties through a combination of advanced AM multiphysics simulations and data-driven surrogate modeling. Model correction and parameter calibration for the constructed surrogate models using limited amounts of experimental data are discussed.

Contribution of the Greenland Ice Sheet to sea level over the next millennium.

The Greenland Ice Sheet holds 7.2 m of sea level equivalent and in recent decades, rising temperatures have led to accelerated mass loss. Current ice margin recession is led by the retreat of outlet glaciers, large rivers of ice ending in narrow fjords that drain the interior. We pair an outlet glacier-resolving ice sheet model with a comprehensive uncertainty quantification to estimate Greenland's contribution to sea level over the next millennium. We find that Greenland could contribute 5 to 33 cm to sea level by 2100, with discharge from outlet glaciers contributing 8 to 45% of total mass loss. Our analysis shows that uncertainties in projecting mass loss are dominated by uncertainties in climate scenarios and surface processes, whereas uncertainties in calving and frontal melt play a minor role. We project that Greenland will very likely become ice free within a millennium without substantial reductions in greenhouse gas emissions.

Effects of left ventricle wall thickness uncertainties on cardiac mechanics.

Computational models of the heart have reached a level of maturity that enables sophisticated patient-specific simulations and hold potential for important applications in diagnosis and therapy planning. However, such clinical use puts strict demands on the reliability and accuracy of the models and requires the sensitivity of the model predictions due to errors and uncertainty in the model inputs to be quantified. The models typically contain a large number of parameters, which are difficult to measure and therefore associated with considerable uncertainty. Additionally, patient-specific geometries are usually constructed by semi-manual processing of medical images and must be assumed to be a potential source of model uncertainty. In this paper, we assess the model accuracy by considering the impact of geometrical uncertainties, which typically occur in image-based computational geometries. An approach based on 17 AHA segments diagram is used to consider uncertainties in wall thickness and also in the material properties and fiber orientation, and we perform a comprehensive uncertainty quantification and sensitivity analysis based on polynomial chaos expansions. The quantities considered include stress, strain and global deformation parameters of the left ventricle. The results indicate that important quantities of interest may be more affected by wall thickness, and highlight the need for accurate geometry reconstructions in patient-specific cardiac mechanics models.

Application of variance-based uncertainty and sensitivity analysis to biological modeling in carbon ion treatment plans.

In ion beam therapy, biological models to estimate the relative biological effectiveness (RBE) and subsequently the RBE-weighted dose (RWD) are needed in treatment planning and plan evaluation. The required biological parameters as well as their dependency on ion species and ion energy can typically not be determined directly in experiments for invivo situations. For that reason they are often derived from invitro data and biological modeling and subject to large uncertainties. We present a model-independent Monte Carlo (variance-) based uncertainty and Sensitivity Analysis (SA) approach to quantify the impact of different input uncertainties on a simulated carbon ion treatment plan. The influences of different input uncertainties are examined by variance-based SA methods. In this Monte Carlo approach, a function is evaluated 103 -105 times. For each of those runs, all inputs are changed simultaneously, using random numbers according to their associated uncertainties. Variance-based statistic formalisms then rank the input parameter/uncertainty pairs according to their impact on the result of the function. The method of SA includes an uncertainty analysis and was applied to a two-field spot scanning carbon ion treatment plan for two commonly used biological models and two representative tissue parameter sets. Based on an exemplary patient case, the application of variance-based SA for biological measures, relevant in (carbon) ion therapy, is demonstrated. A voxel-wise calculation for 2.9·105 voxels takes ~6h. A structure-based SA, which adds an uncertainty band to a RWD-volume histogram (RW-DVH) and shows how to decrease the uncertainty in the most effective way, can be calculated in 0.1-1.5h (depending on the size of the structure). The uncertainties in RBE, RWD or RW-DVH are broken down to the impact of different uncertainties in the (biological) model input. Biological uncertainties have a higher impact on the resulting RBE and RWD than uncertainties in the physical dose. Excluding the physical dose from the SA only slightly decreased the overall uncertainty, emphasizing the necessity to include biological uncertainties into treatment plan evaluation. Variance-based SA is a powerful tool to evaluate the impact of uncertainties in (carbon) ion therapy. The number of input parameters that can be examined at once is only limited by computation time. A Monte Carlo-derived, comprehensive uncertainty quantification and a corresponding sensitivity analysis are implemented and provide new information for treatment plan evaluation. A possible future application is a SA-based biologically robust treatment plan optimization using the additional uncertainty information as presented here.

Evidential uncertainty quantification of the Park–Ang damage model in performance based design

PurposeThis paper aims to develop a comprehensive uncertainty quantification method using evidence theory for Park–Ang damage index-based performance design in which epistemic uncertainties are considered. Various sources of uncertainty emanating from the database of the cyclic test results of RC members provided by the Pacific Earthquake Engineering Research Center are taken into account.Design/methodology/approachIn this paper, an uncertainty quantification methodology based on evidence theory is presented for the whole process of performance-based seismic design (PBSD), while considering uncertainty in the Park–Ang damage model. To alleviate the burden of high computational cost in propagating uncertainty, the differential evolution interval optimization strategy is used for efficiently finding the propagated belief structure throughout the whole design process.FindingsThe investigation results of this paper demonstrate that the uncertainty rooted in Park–Ang damage model have a significant influence on PBSD design and evaluation. It might be worth noting that the epistemic uncertainty present in the Park–Ang damage model needs to be considered to avoid underestimating the true uncertainty.Originality/valueThis paper presents an evidence theory-based uncertainty quantification framework for the whole process of PBSD.

Uncertainty analysis and estimation of robust AIREBO parameters for graphene

AIREBO is one of the commonly utilized interatomic potential (IP) for performing molecular dynamics (MD) simulations of graphene and carbon nanostructures. With its parameters fitted to a limited dataset, property prediction outside of the original training set can be challenging and can lead to uncertainty in the predicted values. This is especially important for 2D materials such as graphene which have limited experimental data and have widely varying predicted properties in the literature. In this study, we conducted a comprehensive Uncertainty Quantification (UQ) analysis of AIREBO potential parameters and their corresponding effect on the predicted properties of graphene. We found that computed output properties were highly sensitive to small variations in IP parameters. For instance, a 0.5% change in IP parameters led to a 66% change in the predicted elastic constants. Based on our UQ analysis, we developed a new robust IP parameter set for the AIREBO potential with significantly reduced sensitivity towards output properties. The robust parameters were derived using a Markov Chain Monte Carlo scheme, considering gaussian noise in available DFT data. We were also able to obtain realistic error bars on MD predictions by using posterior probability distributions and propagating the underlying variance to the final properties.

Identifying Active Sites of the Water–Gas Shift Reaction over Titania Supported Platinum Catalysts under Uncertainty

A comprehensive uncertainty quantification framework has been developed for integrating computational and experimental kinetic data and to identify active sites and reaction mechanisms in catalysis. Three hypotheses regarding the active site for the water–gas shift reaction on Pt/TiO2 catalysts, Pt(111), an edge interface site, and a corner interface site, are tested against experimental kinetic data from three research groups. Uncertainties associated with density functional theory (DFT) calculations and model errors of microkinetic models of the active sites are informed and verified using Bayesian inference and predictive validation. Significant evidence is found for the role of the oxide support in the mechanism. Positive evidence is found in support of the edge interface active site over the corner interface site. For the edge interface site, the CO-promoted redox mechanism is found to be the dominant pathway and only at temperatures above 573 K does the classical redox mechanism contribute significa...

Review of Robust Aerodynamic Design Optimization for Air Vehicles

The ever-increasing demands for risk-free, resource-efficient and environment-friendly air vehicles motivate the development of advanced design methodology. As a particularly promising design methodology considering uncertainties, robust aerodynamic design optimization (RADO) is capable of providing robust and reliable aerodynamic configuration and reducing cost under probable uncertainties in the flight envelop and all life cycle of air vehicle. However, the major challenges including high computational cost with increasing dimensionality of uncertainty and complex RADO procedure hinder the wider application of RADO. In this paper, the complete RADO procedure, i.e., uncertainty modeling, establishment of uncertainty quantification approach as well as robust optimization subject to reliability constraints under uncertainty, is elaborated. Systematic reviews of RADO methodology including uncertainty modeling methods, comprehensive uncertainty quantification approaches, and robust optimization methods are provided. Further, this paper presents a brief survey of the main applications of RADO in the aerodynamic design of transonic flow and natural-laminar-flow, and discusses the application prospects of RADO methodology for air vehicles. The detailed statement of the paper indicates the intention, i.e., to present the state of the art in RADO methodology, to highlight the key techniques and primary challenges in RADO, and to provide the beneficial directions for future researches.

Incorporating a stochastic data‐driven inflow model for uncertainty quantification of wind turbine performance

AbstractIncorporating uncertainty in wind turbine analysis and design is very necessary based on the fact that inherent variability exists in wind turbine systems. Examples of these uncertainties include fluctuations in material properties across turbine blades, variable structure parameters and stochasticity in the inflow—which is considered to be a critical factor affecting the reliability of wind turbines. However, it has been difficult to construct a low‐dimensional yet accurate representation of the stochastic inflow, which precludes rigorous uncertainty propagation and quantification. Recently, we have developed a comprehensive data‐driven approach [called temporal–spatial decomposition (TSD)] for constructing a stochastic, low‐dimensional model that accurately represents stochastic inflow data. We leverage this approach to construct distributional forecasts of key wind turbine performance indicators. To this end, we integrated the stochastic wind model created by the TSD framework with the wind turbine solver FAST. Uncertainty propagation is performed using an adaptive sparse grid collocation approach. We investigate how the order of approximation of the stochastic model affects the quality of the predicted distribution. We observe that the probability distributions of key indicators are not necessarily Gaussian, which has implications for reliability analysis and for failure prediction. Furthermore, the distributions are sensitive to only the first few eigenmodes of the inflow wind model, which indicates that comprehensive uncertainty quantification can potentially be accomplished with moderate computational effort. The approach suggested in this paper enables seamless integration of uncertainty quantification into current deterministic codes for wind turbine simulation and has implications for the design of the next generation of wind turbines including offshore turbines. Copyright © 2017 John Wiley & Sons, Ltd.

Optimal numerical solvers for transient simulations of ice flow using the Ice Sheet System Model (ISSM versions 4.2.5 and 4.11)

Abstract. Identifying fast and robust numerical solvers is a critical issue that needs to be addressed in order to improve projections of polar ice sheets evolving in a changing climate. This work evaluates the impact of using advanced numerical solvers for transient ice-flow simulations conducted with the JPL–UCI Ice Sheet System Model (ISSM). We identify optimal numerical solvers by testing a broad suite of readily available solvers, ranging from direct sparse solvers to preconditioned iterative methods, on the commonly used Ice Sheet Model Intercomparison Project for Higher-Order ice sheet Models benchmark tests. Three types of analyses are considered: mass transport, horizontal stress balance, and incompressibility. The results of the fastest solvers for each analysis type are ranked based on their scalability across mesh size and basal boundary conditions. We find that the fastest iterative solvers are ∼ 1.5–100 times faster than the default direct solver used in ISSM, with speed-ups improving rapidly with increased mesh resolution. We provide a set of recommendations for users in search of efficient solvers to use for transient ice-flow simulations, enabling higher-resolution meshes and faster turnaround time. The end result will be improved transient simulations for short-term, highly resolved forward projections (10–100 year time scale) and also improved long-term paleo-reconstructions using higher-order representations of stresses in the ice. This analysis will also enable a new generation of comprehensive uncertainty quantification assessments of forward sea-level rise projections, which rely heavily on ensemble or sampling approaches that are inherently expensive.

Comprehensive Uncertainty Quantification in Nuclear Safeguards

Nuclear safeguards aim to confirm that nuclear materials and activities are used for peaceful purposes. To ensure that States are honoring their safeguards obligations, quantitative conclusions regarding nuclear material inventories and transfers are needed. Statistical analyses used to support these conclusions require uncertainty quantification (UQ), usually by estimating the relative standard deviation (RSD) in random and systematic errors associated with each measurement method. This paper has two main components. First, it reviews why UQ is needed in nuclear safeguards and examines recent efforts to improve both top-down (empirical) UQ and bottom-up (first-principles) UQ for calibration data. Second, simulation is used to evaluate the impact of uncertainty in measurement error RSDs on estimated nuclear material loss detection probabilities in sequences of measured material balances.