Uncertainty Quantification Methods Research Articles

A comprehensive review of digital twin—part 2: roles of uncertainty quantification and optimization, a battery digital twin, and perspectives

As an emerging technology in the era of Industry 4.0, digital twin is gaining unprecedented attention because of its promise to further optimize process design, quality control, health monitoring, decision- and policy-making, and more, by comprehensively modeling the physical world as a group of interconnected digital models. In a two-part series of papers, we examine the fundamental role of different modeling techniques, twinning enabling technologies, and uncertainty quantification and optimization methods commonly used in digital twins. This second paper presents a literature review of key enabling technologies of digital twins, with an emphasis on uncertainty quantification, optimization methods, open-source datasets and tools, major findings, challenges, and future directions. Discussions focus on current methods of uncertainty quantification and optimization and how they are applied in different dimensions of a digital twin. Additionally, this paper presents a case study where a battery digital twin is constructed and tested to illustrate some of the modeling and twinning methods reviewed in this two-part review. Code and preprocessed data for generating all the results and figures presented in the case study are available on Github.

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network’s latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

Building robust machine learning models for small chemical science data: the case of shear viscosity of fluids

Shear viscosity, though being a fundamental property of all fluids, is computationally expensive to calculate from equilibrium molecular dynamics simulations. Recently, machine learning (ML) methods have been used to augment molecular simulations in many contexts, thus showing promise to estimate viscosity too in a relatively inexpensive manner. However, ML methods face significant challenges—such as overfitting, when the size of the data set is small, as is the case with viscosity. In this work, we train seven ML models to predict the shear viscosity of a Lennard–Jones fluid, with particular emphasis on addressing issues arising from a small data set. Specifically, the issues related to model selection, performance estimation and uncertainty quantification were investigated. First, we show that the widely used performance estimation procedure of using a single unseen data set shows a wide variability—in estimating the errors on—small data sets. In this context, the common practice of using cross validation (CV) to select the hyperparameters (model selection) can be adapted to estimate the generalization error (performance estimation) as well. We compare two simple CV procedures for their ability to do both model selection and performance estimation, and find that k-fold CV based procedure shows a lower variance of error estimates. Also, these CV procedures naturally lead to an ensemble of trained ML models. We discuss the role of performance metrics in training and evaluation and propose a method to rank the ML models based on multiple metrics. Finally, two methods for uncertainty quantification—Gaussian process regression (GPR) and ensemble method—were used to estimate the uncertainty on individual predictions. The uncertainty estimates from GPR were also used to construct an applicability domain using which the ML models provided even more reliable predictions on an independent viscosity data set generated in this work. Overall, the procedures prescribed in this work, together, lead to robust ML models for small data sets.

Learning Groundwater Contaminant Diffusion‐Sorption Processes With a Finite Volume Neural Network

AbstractImproved understanding of complex hydrosystem processes is key to advance water resources research. Nevertheless, the conventional way of modeling these processes suffers from a high conceptual uncertainty, due to almost ubiquitous simplifying assumptions used in model parameterizations/closures. Machine learning (ML) models are considered as a potential alternative, but their generalization abilities remain limited. For example, they normally fail to predict accurately across different boundary conditions. Moreover, as a black box, they do not add to our process understanding or to discover improved parameterizations/closures. To tackle this issue, we propose the hybrid modeling framework FINN (finite volume neural network). It merges existing numerical methods for partial differential equations (PDEs) with the learning abilities of artificial neural networks (ANNs). FINN is applied on discrete control volumes and learns components of the investigated system equations, such as numerical stencils, model parameters, and arbitrary closure/constitutive relations. Consequently, FINN yields highly interpretable results. We demonstrate FINN's potential on a diffusion‐sorption problem in clay. Results on numerically generated data show that FINN outperforms other ML models when tested under modified boundary conditions, and that it can successfully differentiate between the usual, known sorption isotherms. Moreover, we also equip FINN with uncertainty quantification methods to lay open the total uncertainty of scientific learning, and then apply it to a laboratory experiment. The results show that FINN performs better than calibrated PDE‐based models as it is able to flexibly learn and model sorption isotherms without being restricted to choose among available parametric models.

A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies

As an emerging technology in the era of Industry 4.0, digital twin is gaining unprecedented attention because of its promise to further optimize process design, quality control, health monitoring, decision and policy making, and more, by comprehensively modeling the physical world as a group of interconnected digital models. In a two-part series of papers, we examine the fundamental role of different modeling techniques, twinning enabling technologies, and uncertainty quantification and optimization methods commonly used in digital twins. This first paper presents a thorough literature review of digital twin trends across many disciplines currently pursuing this area of research. Then, digital twin modeling and twinning enabling technologies are further analyzed by classifying them into two main categories: physical-to-virtual, and virtual-to-physical, based on the direction in which data flows. Finally, this paper provides perspectives on the trajectory of digital twin technology over the next decade, and introduces a few emerging areas of research which will likely be of great use in future digital twin research. In part two of this review, the role of uncertainty quantification and optimization are discussed, a battery digital twin is demonstrated, and more perspectives on the future of digital twin are shared. Code and preprocessed data for generating all the results and figures presented in the battery digital twin case study in part 2 of this review are available on Github .

Research on the Application of Uncertainty Quantification (UQ) Method in High-Voltage (HV) Cable Fault Location

In HV cable fault location technology, line parameter uncertainty has an impact on the location criterion and affects the fault location result. Therefore, it is of great significance to study the uncertainty quantification of line parameters. In this paper, an impedance-based fault location criterion was used for an uncertainty study. Three kinds of uncertainty factors, namely the sheath resistivity per unit length, the equivalent grounding resistance on both sides, and the length of the cable section, were taken as random input variables without interaction. They were subject to random uniform distribution within a 50% amplitude variation. The relevant statistical information, such as the mean value, standard deviation and probability distribution, of the normal operation and fault state were calculated using the Monte Carlo simulation (MCS) method, the polynomial chaos expansion (PCE) method, and the univariate dimension reduction method (UDRM), respectively. Thus, the influence of uncertain factors on fault location was analyzed, and the calculation results of the three uncertainty quantification methods compared. The results indicate that: (1) UQ methods are effective for simulation analysis of fault locations, and UDRM has certain application prospects for HV fault location in practice; (2) the quantification results of the MCS, PCE, and UDRM were very close, while the mean convergence rate was significantly higher for the UDRM; (3) compared with the MCS, PCE, and UDRM, the PCE and UDRM had higher accuracy, and MCS and UDRM required less running time.

Machine Learning with Gradient-Based Optimization of Nuclear Waste Vitrification with Uncertainties and Constraints

Gekko is an optimization suite in Python that solves optimization problems involving mixed-integer, nonlinear, and differential equations. The purpose of this study is to integrate common Machine Learning (ML) algorithms such as Gaussian Process Regression (GPR), support vector regression (SVR), and artificial neural network (ANN) models into Gekko to solve data based optimization problems. Uncertainty quantification (UQ) is used alongside ML for better decision making. These methods include ensemble methods, model-specific methods, conformal predictions, and the delta method. An optimization problem involving nuclear waste vitrification is presented to demonstrate the benefit of ML in this field. ML models are compared against the current partial quadratic mixture (PQM) model in an optimization problem in Gekko. GPR with conformal uncertainty was chosen as the best substitute model as it had a lower mean squared error of 0.0025 compared to 0.018 and more confidently predicted a higher waste loading of 37.5 wt% compared to 34 wt%. The example problem shows that these tools can be used in similar industry settings where easier use and better performance is needed over classical approaches. Future works with these tools include expanding them with other regression models and UQ methods, and exploration into other optimization problems or dynamic control.

Uncertainty quantification method for elastic wave tomography of concrete structure using interval analysis

This paper presents a non-probabilistic uncertainty quantification method for the elastic wave tomography of concrete structures. An interval analysis model is applied to establish the relationship between the measurement error and identified tomography images. Measurements of parameters such as travel time and wave amplitude are modeled as intervals described by their upper and lower boundaries. Based on the straight-ray assumption of the wave propagation path, the relationship between wave velocity and travel time measurements is described by a set of linear equations. The inversion linear equation is converted into an extended fuzzy linear equation using interval analysis. Subsequently, a simultaneous iterative reconstruction technique is applied to solve this fuzzy linear equation. In contrast to the traditional methods, the obtained tomography images are described in terms of the upper and lower boundaries. Finally, numerical simulation and experimental investigation using several plain concrete models are conducted to verify the proposed approach.

Bayesian parameter estimation of SST model for shock wave-boundary layer interaction flows with different strengths

The Shock Wave-Boundary Layer Interaction (SWBLI) flow generated by compression corner widely occurs in engineering. As one of the primary methods in engineering, the Reynolds Averaged Navier-Stokes (RANS) methods usually cannot correctly predict strong SWBLI flows. In addition to the defects of the eddy viscosity assumption, the uncertainty of the closure coefficients in RANS models often significantly impacts the simulation results. This study performs parametric sensitivity analysis and Bayesian calibration on the closure coefficients of the Menter k-ω Shear-Stress Transport (SST) model based on the SWBLI with different strengths. Firstly, the parametric sensitivity on prediction results is analyzed using the Sobol index. The results indicate that the Sobol indices of wall pressure and skin friction exhibited opposite fluctuation trends with the increase of SWBLI strength. Then, the Bayesian uncertainty quantification method is adopted to obtain the posterior probability distributions and Maximum A Posteriori (MAP) estimates of the closure coefficients and the posterior uncertainty of the Quantities of Interests (QoIs). The results indicate that the prediction ability for strong SWBLI of the SST model is significantly improved by using the MAP estimates, and the relative errors of QoIs are reduced dramatically.

Model Uncertainty and Correctability for Directed Graphical Models

Probabilistic graphical models are a fundamental tool in probabilistic modeling, machine learning and artificial intelligence. They allow us to integrate in a natural way expert knowledge, physical modeling, heterogeneous and correlated data and quantities of interest. For exactly this reason, multiple sources of model uncertainty are inherent within the modular structure of the graphical model. In this paper we develop information-theoretic, robust uncertainty quantification methods and non-parametric stress tests for directed graphical models to assess the effect and the propagation through the graph of multi-sourced model uncertainties to quantities of interest. These methods allow us to rank the different sources of uncertainty and correct the graphical model by targeting its most impactful components with respect to the quantities of interest. Thus, from a machine learning perspective, we provide a mathematically rigorous approach to correctability that guarantees a systematic selection for improvement of components of a graphical model while controlling potential new errors created in the process in other parts of the model. We demonstrate our methods in two physico-chemical examples, namely quantum scale-informed chemical kinetics and materials screening to improve the efficiency of fuel cells.

Novel Nonprobabilistic Bayesian Uncertainty Quantification Method for Structures with Interval Parameters

Structural reliability analysis is a crucial task in structural safety analysis. Owing to insufficient data information, traditional probabilistic methods do not accurately quantify input or output variables and the resultant assessment of structural reliability. Considering the development of engineering technology, a quantitative model of uncertainty based on improbability method is urgently needed. Herein, a novel nonprobabilistic Bayesian-inference-based interval uncertainty quantification model is developed with a user-specified credibility level. On the contrary, limited samples are introduced to update the bounds of the interval, and the conservativeness is decreased compared to the traditional unbiased interval estimation based on uniform distribution. Moreover, it can provide more conservative quantification than the extreme values-based interval model. Consequently, two examples are implemented to demonstrate the feasibility and effectiveness of the proposed model.

Comparison of inverse uncertainty quantification methods for critical flow test

A problem of epistemic uncertainties introduced by code input parameters that cannot be estimated other than by user expertise has existed in nuclear reactor systems analyses since first uncertainty analyses were conducted. Inverse Uncertainty Quantification (IUQ) methods aim at providing an estimation of the distributions of such parameters. This paper compares and assesses two approaches to quantify the uncertainty of critical flow model parameters available in the TRACE code. First novel approach is based on a machine-learning algorithm, using a Random Forest classifier to assign the results of calculations to one of the defined classes of prediction accuracy with respect to experimental data. The second approach is based on Markov Chain Monte Carlo sampling and Bayesian inference. Both methods allows to assign probability distribution functions to TRACE internal variables, which is a goal for IUQ methods.Both approaches were used to estimate uncertainties for four critical flow parameters available in the TRACE code based on the Marviken critical flow experiment. In case of two parameters both methods produced similar results of the probability density functions. Based on the results of machine learning two additional parameters were deemed noninfluential, while for the second method such definitive assessment could not be made.

Diversity entropy-based Bayesian deep learning method for uncertainty quantification of remaining useful life prediction in rolling bearings

Remaining useful life (RUL) prediction of rolling bearings plays a critical role in reducing unplanned downtime and improving machine productivity. The existing prediction methods primarily provide point estimates of RUL without quantifying uncertainty. However, uncertainty quantification of RUL is crucial to conduct reliable risk analysis and make maintenance decision, which can significantly decrease the maintenance costs. To solve the uncertainty quantification problem and improve prediction accuracy at the same time, a novel diversity entropy-based Bayesian deep learning (DE-BDL) method is proposed. First, start degradation time (SDT) of bearings is adaptively determined using diversity entropy, which can extract early degradation information. Then, multi-scale diversity entropy (MDE) is developed to extract dynamic characteristics over multiple scales. Third, the obtained features using MDE are fed into the BDL model for degradation tracking and prediction. By doing this, the proposed DE-BDL method has merits in subsequent decision making, which can not only provide point estimation but also offer uncertainty quantification with epistemic uncertainty and aleatoric uncertainty. The superiority of the proposed method is validated using run-to-failure data. The experimental results and comparison with state-of-art prediction methods have demonstrated that the proposed DE-BDL method is promising for RUL of rolling bearings.

Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes

Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, non-stationary kernel designs can be defined in the same framework to yield flexible multi-task Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results show that including domain knowledge, communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.

Structural Health Monitoring Impact Classification Method Based on Bayesian Neural Network

This paper proposes a novel method for multi-class classification and uncertainty quantification of impact events on a flat composite plate with a structural health monitoring (SHM) system by using a Bayesian neural network (BNN). Most of the existing research in passive sensing has focused on deterministic approaches for impact detection and characterization. However, there are variability in impact location, angle and energy in real operational conditions which results in uncertainty in the diagnosis. Therefore, this paper proposes a reliability-based impact characterization method based on BNN for the first time. Impact data are acquired by a passive sensing system of piezoelectric (PZT) sensors. Features extracted from the sensor signals, such as their transferred energy, frequency at maximum amplitude and time interval of the largest peak, are used to develop a BNN for impact classification (i.e., energy level). To test the robustness and reliability of the proposed model to impact variability, it is trained with perpendicular impacts and tested by variable angle impacts. The same dataset is further applied in a method called multi-artificial neural network (multi-ANN) to compare its ability in uncertainty quantification and its computational efficiency against the BNN for validation of the developed meta-model. It is demonstrated that both the BNN and multi-ANN can measure the uncertainty and confidence of the diagnosis from the prediction results. Both have very high performance in classifying impact energies when the networks are trained and tested with perpendicular impacts of different energy and location, with 94% and 98% reliable predictions for BNN and multi-ANN, respectively. However, both metamodels struggled to detect new impact scenarios (angled impacts) when the data set was not used in the development stage and only used for testing. Including additional features improved the performance of the networks in regularization; however, not to the acceptable accuracy. The BNN significantly outperforms the multi-ANN in computational time and resources. For perpendicular impacts, both methods can reach a reliable accuracy, while for angled impacts, the accuracy decreases but the uncertainty provides additional information that can be further used to improve the classification.

Deep adaptive arbitrary polynomial chaos expansion: A mini-data-driven semi-supervised method for uncertainty quantification

All kinds of uncertainties influence the reliability of the engineering system. Thus, uncertainty quantification is significant to the system reliability analysis. Polynomial chaos expansion (PCE) is an effective method for uncertainty quantification while it requires sufficient labeled data to quantify uncertainty accurately. To overcome this problem, this paper proposes the adaptive arbitrary polynomial chaos (aPC) and proves two properties of the adaptive expansion coefficients. Sequentially, a semi-supervised deep adaptive arbitrary polynomial chaos expansion (Deep aPCE) method is proposed based on the adaptive aPC and the deep neural network (DNN). The Deep aPCE method uses two properties of the adaptive aPC to assist in training the DNN by a small amount of labeled data and abundant unlabeled data, significantly reducing the training data cost. On the other hand, the Deep aPCE method adopts the DNN to fine-tune the adaptive expansion coefficients dynamically, improving the accuracy of uncertainty quantification. Besides, the Deep aPCE method can directly construct accurate surrogate models of the high dimensional stochastic systems. Five numerical examples are used to verify the effectiveness of the Deep aPCE method. Finally, the Deep aPCE method is applied to the reliability analysis of an axisymmetric conical aircraft.

A generalized probabilistic learning approach for multi-fidelity uncertainty quantification in complex physical simulations

Two of the most significant challenges in uncertainty quantification pertain to the high computational cost for simulating complex physical models and the high dimension of the random inputs. In applications of practical interest, both of these problems are encountered, and standard methods either fail or are not feasible. To overcome the current limitations, we present a generalized formulation of a Bayesian multi-fidelity Monte-Carlo (BMFMC) framework that can exploit lower-fidelity model versions in a small data regime. The goal of our analysis is an efficient and accurate estimation of the complete probabilistic response for high-fidelity models. BMFMC circumvents the curse of dimensionality by learning the relationship between the outputs of a reference high-fidelity model and potentially several lower-fidelity models. While the continuous formulation is mathematically exact and independent of the low-fidelity model’s accuracy, we address challenges associated with the small data regime (i.e., only a small number of 50 to 300 high-fidelity model runs can be performed). Specifically, we complement the formulation with a set of informative input features at no extra cost. Despite the inaccurate and noisy information that some low-fidelity models provide, we demonstrate that accurate and certifiable estimates for the quantities of interest can be obtained for uncertainty quantification problems in high stochastic dimensions, with significantly fewer high-fidelity model runs than state-of-the-art methods for uncertainty quantification. We illustrate our approach by applying it to challenging numerical examples such as Navier–Stokes flow simulations and fluid–structure interaction problems.

A review on uncertainty quantification of shadowing reconstruction and signal measurements in Radio Tomographic Imaging

Radio Tomographic Imaging (RTI) is a novel technology to reconstruct the target-induced shadowing effect of signal propagation in the Radio-Frequency (RF) sensing network, through the Received Signal Strength (RSS) measurements in the RF propagation links. As a novel wireless-signal reconstruction technology, RTI is widely utilized in the Device-Free Localization (DFL) and Location-Based Services (LBS) applications. However, the multipath interference in RF sensing network often induces serious uncertainty in RTI system, including the RSS measurement noise and the RTI reconstruction degradation. Addressing this problem, as the RTI system is to reconstruct the probability image of shadowing by using the RSS measurements, from the perspective of probabilistic modeling, the RTI reconstruction issue is modeled as the Maximum a Posterior Probability (MAP) of the reconstructed shadowing image from the RSS measurements. Then lots of RTI reconstruction methods for uncertainty quantification, have been developed to improve the RSS measurement accuracy and the RTI reconstruction quality. Therefore, for the first time, this article conducts a comprehensive survey of the recent RTI methods from the viewpoint of uncertainty quantification. At first, the uncertainty quantification in these RTI methods of the shadowing image and RSS measurements are united into the Epistemic uncertainty and Aleatoric uncertainty respectively. Besides, these methods are further classified into different categories based on their technical commonalities. Moreover, the presenting challenges of uncertainty quantification in the actual RTI applications are also discussed, and based on that, the potential development directions of these methods for improving the RSS measurement accuracy and RTI reconstruction performance in future are reasonably predicted.

Adjoint-based uncertainty quantification for inhomogeneous friction on a slow-slipping fault

SUMMARY Long-term slow-slip events (LSSEs) usually occur on a fault existing at the deep, shallow parts of subducting plates and substantially relate to adjacent megathrust fault motions. The dynamics of the LSSE largely depend on the inhomogeneity of friction that occurs between the fault interfaces. Thus, it is crucial to estimate the spatial-dependent frictional features from the observations of the slip motion and subsequently identify essential parts that contribute to the principal slip motion by quantifying uncertainties involved in the estimates. Although quantifying the uncertainties of the frictional feature fields in high resolution is necessary to solve the task, conventional techniques of quantifying slow earthquake frictional features have not yet achieved such uncertainty quantification (UQ) due to the complexity of LSSE models such as the large dimensionality. We, therefore, propose a method of UQ for spatially inhomogeneous frictional features from slip motion based on a 4-D variational data assimilation technique using a second-order adjoint method. The proposed method enables us to conduct an accurate UQ even when the dimensionality is large. By combining a fault motion model that mimics slow-slip motion on an LSSE fault–megathrust fault complex in southwestern Japan and the data assimilation technique, we successfully quantified the spatial distribution of the uncertainty of the frictional features in high-resolution. The evaluated spatial distribution in high-resolution reveals the correlation between the dynamics of the slow-slip motion and the important components of the frictional features, which is a valuable information for designing observation systems. Findings from this study are expected to advance the theoretical foundation of applied seismic motion prediction techniques using slow-slip frictional features as stress metres for megaquakes, as well as to improve the understanding of the relationship between the slow-slip motion and frictional parameters of a fault.

Uncertainty quantification for ecological models with random parameters.

There is often considerable uncertainty in parameters in ecological models. This uncertainty can be incorporated into models by treating parameters as random variables with distributions, rather than fixed quantities. Recent advances in uncertainty quantification methods, such as polynomial chaos approaches, allow for the analysis of models with random parameters. We introduce these methods with a motivating case study of sea ice algal blooms in heterogeneous environments. We compare Monte Carlo methods with polynomial chaos techniques to help understand the dynamics of an algal bloom model with random parameters. Modelling key parameters in the algal bloom model as random variables changes the timing, intensity and overall productivity of the modelled bloom. The computational efficiency of polynomial chaos methods provides a promising avenue for the broader inclusion of parametric uncertainty in ecological models, leading to improved model predictions and synthesis between models and data.