Accurate Uncertainty Quantification Research Articles

A Bayesian-Based Credible and Reliable Control Method for Active Vibration Suppression of Structures with Uncertain Parameters

As a classical design method for addressing uncertainty in control systems, robust control is designed with the goal of complete reliability of the control system. Nevertheless, the pursuit of robustness can inadvertently give rise to considerable performance degradation, rendering the resultant closed-loop control system excessively conservative in its design. Reliability-based design methods serve as an effective approach for managing uncertainties, which will achieve a balance between the robustness and performance of the active control system by taking advantage of the characteristic of reliability that allows for a certain level of permissible failure probability. The accurate quantification of uncertainty is the key to achieving reliable control, but common uncertainty quantification methods cannot achieve credible quantification of uncertainty in small samples and update the distribution of new samples. In this paper, a nonprobabilistic Bayesian inference based interval uncertainty quantification method is proposed. This method updates the distribution of interval boundaries and radii by introducing a small number of samples, and obtains interval parameters under a given confidence level. On the basis of uncertainty quantification, the dynamic reliability of the vibration system is derived, and nonprobabilistic reliability is introduced into the design of active controllers through optimization strategies. The reliable control strategy aims to select the optimal controller parameters on a reliable path with a given credibility, in order to solve the problem of overly conservative robust control theory. Two numerical examples were used to verify the feasibility and applicability of this method.

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Enhancing trustworthiness in ML-based network intrusion detection with uncertainty quantification

A crucial role in the security of modern networks is played by Intrusion Detection Systems (IDSs), security devices designed to identify and mitigate attacks to networks structure. Data-driven approaches based on Machine Learning (ML) have gained more and more popularity for executing the classification tasks required by signature-based IDSs. However, typical ML models adopted for this purpose do not properly take into account the uncertainty associated with their prediction. This poses significant challenges, as they tend to produce misleadingly high classification scores for both misclassified inputs and inputs belonging to unknown classes (e.g. novel attacks), limiting the trustworthiness of existing ML-based solutions. In this paper, we argue that ML-based IDSs should always provide accurate uncertainty quantification to avoid overconfident predictions. In fact, an uncertainty-aware classification would be beneficial to enhance closed-set classification performance, would make it possible to carry out Active Learning, and would help recognize inputs of unknown classes as truly unknowns, unlocking open-set classification capabilities and Out-of-Distribution (OoD) detection. To verify it, we compare various ML-based methods for uncertainty quantification and open-set classification, either specifically designed for or tailored to the domain of network intrusion detection. Moreover, we develop a custom model based on Bayesian Neural Networks that stands out for its OoD detection capabilities and robustness, with a lower variance in the results over different scenarios, compared to other baselines, thus showing how proper uncertainty quantification can be exploited to significantly enhance the trustworthiness of ML-based IDSs.

Development and Verification of Sampling Code NECP-SOUL for Evaluated Nuclear Data Files in ENDF-6 Format

A code called NECP-SOUL is developed to generate samples of evaluated nuclear data files in the ENDF-6 format. The motivation for the development is to obtain more accurate uncertainty quantification results based on the covariance data given in evaluated nuclear data files than the sampling method based on multigroup covariances. NECP-SOUL can handle all covariance data in ENDF-6 files, including average fission neutron multiplicities, resonance parameters, cross sections, angular distributions, energy distributions, and cross sections for the production of radioactive nuclides. In this work, samples of ENDF-6 files are input to the nuclear data processing code to generate A Compact ENDF (ACE)-formatted libraries, and these ACE-formatted libraries are used by the Monte Carlo code. Through statistical analysis, the uncertainty of the neutronics results can be obtained. NECP-SOUL is verified against another similar code, SANDY. In addition, the advantage of sampling ENDF-6 files is analyzed by comparing the uncertainty quantification results with those obtained by sampling multigroup covariance matrices. Moreover, the covariances of cross sections and angular distributions newly provided in ENDF/B-VIII.0 are used to analyze their effect on the uncertainty qualification results.

Ultra-Efficient MCMC for Bayesian Longitudinal Functional Data Analysis

Functional mixed models are widely useful for regression analysis with dependent functional data, including longitudinal functional data with scalar predictors. However, existing algorithms for Bayesian inference with these models only provide either scalable computing or accurate approximations to the posterior distribution, but not both. We introduce a new MCMC sampling strategy for highly efficient and fully Bayesian regression with longitudinal functional data. Using a novel blocking structure paired with an orthogonalized basis reparameterization, our algorithm jointly samples the fixed effects regression functions together with all subject- and replicate-specific random effects functions. Crucially, the joint sampler optimizes sampling efficiency for these key parameters while preserving computational scalability. Perhaps surprisingly, our new MCMC sampling algorithm even surpasses state-of-the-art algorithms for frequentist estimation and variational Bayes approximations for functional mixed models—while also providing accurate posterior uncertainty quantification—and is orders of magnitude faster than existing Gibbs samplers. Simulation studies show improved point estimation and interval coverage in nearly all simulation settings over competing approaches. We apply our method to a large physical activity dataset to study how various demographic and health factors associate with intraday activity. Supplementary materials for this article are available online.

Application of simultaneous uncertainty quantification and segmentation for oropharyngeal cancer use-case with Bayesian deep learning

BackgroundRadiotherapy is a core treatment modality for oropharyngeal cancer (OPC), where the primary gross tumor volume (GTVp) is manually segmented with high interobserver variability. This calls for reliable and trustworthy automated tools in clinician workflow. Therefore, accurate uncertainty quantification and its downstream utilization is critical.MethodsHere we propose uncertainty-aware deep learning for OPC GTVp segmentation, and illustrate the utility of uncertainty in multiple applications. We examine two Bayesian deep learning (BDL) models and eight uncertainty measures, and utilize a large multi-institute dataset of 292 PET/CT scans to systematically analyze our approach.ResultsWe show that our uncertainty-based approach accurately predicts the quality of the deep learning segmentation in 86.6% of cases, identifies low performance cases for semi-automated correction, and visualizes regions of the scans where the segmentations likely fail.ConclusionsOur BDL-based analysis provides a first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.

Efficient and accurate uncertainty quantification in engineering simulations using time-separated stochastic mechanics

A robust method for uncertainty quantification is undeniably leading to a greater certainty in simulation results and more sustainable designs. The inherent uncertainties of the world around us render everything stochastic, from material parameters, over geometries, up to forces. Consequently, the results of engineering simulations should reflect this randomness. Many methods have been developed for uncertainty quantification for linear elastic material behavior. However, real-life structure often exhibit inelastic material behavior such as visco-plasticity. Inelastic material behavior is described by additional internal variables with accompanying differential equations. This increases the complexity for the computation of stochastic quantities, e.g., expectation and standard deviation, drastically. The time-separated stochastic mechanics is a novel method for the uncertainty quantification of inelastic materials. It is based on a separation of all fields into a sum of products of time-dependent but deterministic and stochastic but time-independent terms. Only a low number of deterministic finite element simulations are then required to track the effect of (in)homogeneous material fluctuations on stress and internal variables. Despite the low computational effort the results are often indistinguishable from reference Monte Carlo simulations for a variety of boundary conditions and loading scenarios.

Enhancing Probabilistic Solar PV Forecasting: Integrating the NB-DST Method with Deterministic Models

Accurate quantification of uncertainty in solar photovoltaic (PV) generation forecasts is imperative for the efficient and reliable operation of the power grid. In this paper, a data-driven non-parametric probabilistic method based on the Naïve Bayes (NB) classification algorithm and Dempster–Shafer theory (DST) of evidence is proposed for day-ahead probabilistic PV power forecasting. This NB-DST method extends traditional deterministic solar PV forecasting methods by quantifying the uncertainty of their forecasts by estimating the cumulative distribution functions (CDFs) of their forecast errors and forecast variables. The statistical performance of this method is compared with the analog ensemble method and the persistence ensemble method under three different weather conditions using real-world data. The study results reveal that the proposed NB-DST method coupled with an artificial neural network model outperforms the other methods in that its estimated CDFs have lower spread, higher reliability, and sharper probabilistic forecasts with better accuracy.

Novel Entropy for Enhanced Thermal Imaging and Uncertainty Quantification.

This paper addresses the critical need for precise thermal modeling in electronics, where temperature significantly impacts system reliability. We emphasize the necessity of accurate temperature measurement and uncertainty quantification in thermal imaging, a vital tool across multiple industries. Current mathematical models and uncertainty measures, such as Rényi and Shannon entropies, are inadequate for the detailed informational content required in thermal images. Our work introduces a novel entropy that effectively captures the informational content of thermal images by combining local and global data, surpassing existing metrics. Validated by rigorous experimentation, this method enhances thermal images' reliability and information preservation. We also present two enhancement frameworks that integrate an optimized genetic algorithm and image fusion techniques, improving image quality by reducing artifacts and enhancing contrast. These advancements offer significant contributions to thermal imaging and uncertainty quantification, with broad applications in various sectors.

Improved hierarchical Bayesian modeling framework with arbitrary polynomial chaos for probabilistic model updating

Bayesian finite element model updating techniques have found widespread application in the structural health monitoring. The conventional Bayesian modeling framework (CBMF) identifies the posterior distribution of structural parameters using data from a single experiment. However, structural parameters exhibit variability due to changes in environmental or experimental conditions, an aspect overlooked by CBMF. In recent years, the hierarchical Bayesian modeling framework (HBMF) has gradually attracted attention. This framework incorporates an additional layer of hyperparameters and utilizes multiple sets of experimental data to describe the variability in structural parameters caused by changes in experimental conditions. Unfortunately, the increase in hyperparameters and data size leads the HBMF to face the dilemma of exceedingly high computational costs. To address this issue, this study proposes an improved hierarchical Bayesian modeling framework (IHBMF). To enhance the computational efficiency of IHBMF, the importance sampling method is employed to simplify the computation of the likelihood function for hyperparameters. Subsequently, a surrogate model based on arbitrary polynomial chaos expansion is introduced to further reduce the computational cost of IHBMF. The framework's accuracy and efficiency were validated through analysis of a simply supported beam and a steel pedestrian bridge. The results demonstrate that IHBMF not only delivers a more accurate quantification of structural parameter uncertainty than CBMF but also exhibits higher computational efficiency compared to HBMF, showcasing its superior capability in structural health monitoring applications.

Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks

Real-world data typically exhibit aleatoric uncertainty which has to be considered during data-driven decision-making to assess the confidence of the decision provided by machine learning models. To propagate aleatoric uncertainty represented by probability distributions (PDs) through neural networks (NNs), both sampling-based and function approximation-based methods have been proposed. However, these methods suffer from significant approximation errors and are not able to accurately represent predictive uncertainty in the NN output. In this paper, we present a novel method, Piecewise Linear Transformation (PLT), for propagating PDs through NNs with piecewise linear activation functions (e.g., ReLU NNs). PLT does not require sampling or specific assumptions about the PDs. Instead, it harnesses the piecewise linear structure of such NNs to determine the propagated PD in the output space. In this way, PLT supports the accurate quantification of predictive uncertainty based on the criterion exactness of the propagated PD. We assess this exactness in theory by showing error bounds for our propagated PD. Further, our experimental evaluation validates that PLT outperforms competing methods on publicly available real-world classification and regression datasets regarding exactness. Thus, the PDs propagated by PLT allow to assess the uncertainty of the provided decisions, offering valuable support.

Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution

Deep learning-based surrogate models have demonstrated remarkable advantages over classical solvers in terms of speed, often achieving speedups of 10 to 1000 times over traditional partial differential equation (PDE) solvers. However, a significant challenge hindering their widespread adoption in both scientific and industrial domains is the lack of understanding about their prediction uncertainties, particularly in scenarios that involve critical decision making. To address this limitation, we propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method, termed Latent Evolution of PDEs with Uncertainty Quantification (LE-PDE-UQ), endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. LE-PDE-UQ leverages latent vectors within a latent space to evolve both the system's state and its corresponding uncertainty estimation. The latent vectors are decoded to provide predictions for the system's state as well as estimates of its uncertainty. In extensive experiments, we demonstrate the accurate uncertainty quantification performance of our approach, surpassing that of strong baselines including deep ensembles, Bayesian neural network layers, and dropout. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions. Our code is available at: https://github.com/AI4Science-WestlakeU/le-pde-uq.

A comprehensive study on seismic dynamic responses of stochastic structures using sparse grid-based polynomial chaos expansion

The seismic response is a critical aspect of structural engineering, as it directly influences the design and safety evaluation of such systems. Structural seismic response becomes more complicated when unavoidable uncertainties in the structure are considered. Therefore, solving and analyzing the stochastic dynamic response of structures efficiently and accurately is of great significance. Various uncertainty quantification methods such as the stochastic finite element method, Monte Carlo method, and polynomial chaos expansion method have made significant progress in this field. However, these methods do not provide a comprehensive study of the seismic response of stochastic structures and efficiency problems rapidly emerge as the number of random variables increases. This paper delivers a comprehensive study of the seismic dynamic response of space truss and reticulated shell structure considering uncertainties in material properties, geometry, and loading. To this end, the finite element model of the structure was first modeled and the deterministic solutions were obtained. Then, polynomial surrogates are modeled for the natural frequency, mode participation factor, mode superposition response, and transient response of the structure under stochastic uncertainties. The sparse grid technique is used to select sparse quadrature points, which overcomes the curse of dimensionality caused by full grid integration. The results demonstrate that the sparse grid-based polynomial chaos expansion is more efficient. Uncertainty quantification reveals the uncertainty of modal analysis, response spectrum analysis, and transient response analysis of space truss and reticulated shell under seismic excitation. In summary, this study provides valuable insights and an efficient and accurate uncertainty quantification tool for the seismic dynamic analysis of stochastic structures, ultimately contributing to safer and more robust structural designs.

Learnable quantile polynomial chaos expansion: An uncertainty quantification method for interval reliability analysis

Various uncertainties like random input and data noise in aerospace engineering affect flight vehicle safety, making uncertainty quantification important for improving system reliability. Polynomial chaos expansion (PCE) is a versatile method for quantifying stochastic systems’ uncertainty caused by random input, while it cannot quantify data uncertainty caused by noise. To solve this problem, a novel uncertainty quantification method, learnable quantile PCE (LQPCE), is proposed for interval reliability analysis. Unlike the traditional PCE, the multi-dimensional orthonormal basis is constructed for combination input rather than only random variables, and the expansion coefficients are parameterized to learnable weights, ensuring the LQPCE model can capture data noise’s features. The LQPCE Pinball loss function is constructed to learn parameterized expansion coefficients by quantile level iterative sampling, achieving accurate prediction and data uncertainty quantification. The quantified data uncertainty is used for interval reliability analysis, calculating more reliable results about the influence of input uncertainty on stochastic systems’ reliability. Three numerical examples and one engineering application validate the LQPCE method’s effectiveness. The results show that the LQPCE method can quantify data uncertainty accurately and obtain credible interval reliability.

Deep energy-pressure regression for a thermodynamically consistent EOS model

In this paper, we aim to explore novel machine learning (ML) techniques to facilitate and accelerate the construction of universal equation-Of-State (EOS) models with a high accuracy while ensuring important thermodynamic consistency. When applying ML to fit a universal EOS model, there are two key requirements: (1) a high prediction accuracy to ensure precise estimation of relevant physics properties and (2) physical interpretability to support important physics-related downstream applications. We first identify a set of fundamental challenges from the accuracy perspective, including an extremely wide range of input/output space and highly sparse training data. We demonstrate that while a neural network (NN) model may fit the EOS data well, the black-box nature makes it difficult to provide physically interpretable results, leading to weak accountability of prediction results outside the training range and lack of guarantee to meet important thermodynamic consistency constraints. To this end, we propose a principled deep regression model that can be trained following a meta-learning style to predict the desired quantities with a high accuracy using scarce training data. We further introduce a uniquely designed kernel-based regularizer for accurate uncertainty quantification. An ensemble technique is leveraged to battle model overfitting with improved prediction stability. Auto-differentiation is conducted to verify that necessary thermodynamic consistency conditions are maintained. Our evaluation results show an excellent fit of the EOS table and the predicted values are ready to use for important physics-related tasks.

A hybrid prognosis scheme for rolling bearings based on a novel health indicator and nonlinear Wiener process

This paper proposes a novel hybrid method aiming at the fault prognosis of bearings. A nonlinear health indicator (HI) is first constructed using Complete Ensemble Empirical Mode Decomposition with Adaptive Noise and Kernel Principal Component Analysis to reflect the health state of a bearing accurately and convincingly. Subsequently, multi-domain features are extracted from vibration signals and the Dual-Channel Transformer Network with the Convolutional Block Attention Module is applied for constructing HIs of the rest bearings. Moreover, the 3σ criterion is employed to establish the condition monitoring interval of health state and detect the First Prediction Time, with which degradation modeling and probabilistic Remaining Useful Life (RUL) prediction are conducted with the assistance of nonlinear Wiener process with random effects. The superior performance of the proposed hybrid prognostic method confirms that the method contributes to the accurate RUL prediction and uncertainty quantification.

AI4OPT: AI Institute for Advances in Optimization

AbstractThis article is a short introduction to AI4OPT, the NSF AI Institute for Advances in Optimization. AI4OPT fuses AI and optimization, inspired by societal challenges in supply chains, energy systems, chip design and manufacturing, and sustainable food systems. By combining machine learning and mathematical optimization, AI4OPT strives to develop AI‐assisted optimization systems that bring orders of magnitude improvements in efficiency, perform accurate uncertainty quantification, and address challenges in resiliency and sustainability. AI4OPT also applies its “teaching the teachers” philosophy to provide longitudinal educational pathways in AI for engineering.

Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems

Machine learning methods for the construction of data-driven reduced order model models are used in an increasing variety of engineering domains, especially as a supplement to expensive computational fluid dynamics for design problems. An important check on the reliability of surrogate models is Uncertainty Quantification (UQ), a self assessed estimate of the model error. Accurate UQ allows for cost savings by reducing both the required size of training data sets and the required safety factors, while poor UQ prevents users from confidently relying on model predictions. We introduce two quality measures for the quantification of uncertainty that are suitable for high dimensional problems: the distribution of normalized residuals on validation data and the distribution of estimated uncertainties. While the distribution of estimated uncertainties evaluates how confident the model is in making predictions, the distribution of normalized residuals evaluates how accurate these uncertainty estimates are We examine several machine learning techniques, including both Gaussian processes and a family UQ-augmented neural networks: Ensemble neural networks (ENN), Bayesian neural networks (BNN), Dropout neural networks (D-NN), and Gaussian neural networks (G-NN). We evaluate UQ accuracy (distinct from model accuracy) We apply these metrics to two model data sets representative of complex dynamical systems: an ocean engineering problem in which a ship traverses irregular wave episodes, and a dispersive wave turbulence system with extreme events, the Majda–McLaughlin–Tabak model. We present conclusions concerning model architecture and hyperparameter tuning for use in applications of UQ to dynamical systems, such as active learning schemes.

Novel numerical method for uncertainty analysis of coupled vibro-acoustic problem considering thermal stress

Uncertainty is pervasive and exerts a profound influence in engineering practice. Uncertainty quantification is a primary task and of paramount importance in uncertainty analysis. To deal with both dependent and independent uncertain parameters in engineering problems, an improved multidimensional parallelepiped model (IMPM) is subsequently developed. Compared with traditional methods that entirely based on statistical features, more accurate uncertainty quantification results can be obtained by means of IMPM. To enhance the efficiency of uncertainty propagation analysis under IMPM, a radial basis function neural network (RBFNN) surrogate model is constructed to replace the original time-consuming finite element simulation model. Eventually, the effectiveness of proposed models and numerical methods is validated through an engineering example involving the coupled vibro-acoustic system considering thermal stress of a hypersonic vehicle.

Fast uncertainty quantification of spent nuclear fuel with neural networks

The accurate calculation and uncertainty quantification of the characteristics of spent nuclear fuel (SNF) play a crucial role in ensuring the safety, efficiency, and sustainability of nuclear energy production, waste management, and nuclear safeguards. State of the art physics-based models, while reliable, are computationally intensive and time-consuming. This paper presents a surrogate modeling approach using neural networks (NN) to predict a number of SNF characteristics with reduced computational costs compared to physics-based models. An NN is trained using data generated from CASMO5 lattice calculations. The trained NN accurately predicts decay heat and nuclide concentrations of SNF, as a function of key input parameters, such as enrichment, burnup, cooling time between cycles, mean boron concentration and fuel temperature. The model is validated against physics-based decay heat simulations and measurements of different uranium oxide fuel assemblies from two different pressurized water reactors. In addition, the NN is used to perform sensitivity analysis and uncertainty quantification. The results are in very good alignment to CASMO5, while the computational costs (taking into account the costs of generating training samples) are reduced by a factor of 10 or more. Our findings demonstrate the feasibility of using NNs as surrogate models for fast characterization of SNF, providing a promising avenue for improving computational efficiency in assessing nuclear fuel behavior and associated risks.