Uncertainty Quantification Research Articles

Fair prediction sets through multi-objective hyperparameter optimization

The widespread implementation of machine learning in safety-critical domains has raised ethical concerns regarding algorithmic discrimination. In such settings, the integration of fairness-aware algorithms with uncertainty quantification tools enables the development of reliable and safe decision-making. In this paper, we introduce a novel methodology that combines conformal prediction, offering rigorous prediction sets, with multi-objective optimization via evolutionary learning. The proposed meta-algorithm optimizes the hyperparameter configuration of classifiers to produce confidence predictors that balance efficiency and equalized coverage guarantees, addressing fairness concerns related to sensitive attributes. We empirically evaluate our methodology with four real-world problems and demonstrate its efficacy in exploring this trade-off and producing a repertoire of Pareto optimal conformal predictors. In this way, our contribution offers different modeling alternatives from which to choose depending on the policy adopted by stakeholders, thus illustrating its capability to enhance equitable decision-making.

High-Confidence Classification of Partial Discharge Acoustic Signals Using Bayesian Networks for Uncertainty Quantification

High-Confidence Classification of Partial Discharge Acoustic Signals Using Bayesian Networks for Uncertainty Quantification

Advanced uncertainty quantification in rainfall-intensity duration frequency curve modeling: A case study of Hilla City and surrounding regions, Iraq

Advanced uncertainty quantification in rainfall-intensity duration frequency curve modeling: A case study of Hilla City and surrounding regions, Iraq

Uncertainty quantification of rock mass discontinuity information and mechanical parameter derived from photogrammetry and Monte Carlo simulation: a case study

Uncertainty quantification of rock mass discontinuity information and mechanical parameter derived from photogrammetry and Monte Carlo simulation: a case study

Quantification of micro- and nano-plastics in atmospheric fine particles by pyrolysis-gas chromatography-mass spectrometry with chromatographic peak reconstruction.

The effects of micro- and nano-plastics (MNPs) on human health are of global concern because MNPs are ubiquitous, persistent, and potentially toxic, particularly when bound to atmospheric fine particles (PM2.5). Traditional quantitative analysis of MNPs by pyrolysis-gas chromatography-mass spectrometry (Py-GC/MS) is often inaccurate because of false positive signals caused by similar polymers and organic compounds. In this study, a reliable analytical strategy combining HNO3 digestion and chromatographic peak reconstruction was developed to improve the precision of pyrolysis-gas chromatography-mass spectrometry analysis of multiple MNPs bound to PM2.5. The optimized HNO3 digestion method using high-pressure oxidation conditions effectively removed organic matter within two hours, giving recovery rates of 64 %-110 % for eight target MNPs. The chromatographic peak reconstruction procedure minimized interferences caused by similar polymers and achieved high accuracy (101 % ± 10 %) for polyvinyl chloride, polyethylene terephthalate, and polystyrene, whose concentrations are often overestimated due to overlapping pyrolysis products. Quantification uncertainties for MNPs in real PM2.5 samples were up to 52 % lower using the new method than using previous methods. The method was validated using PM2.5 from urban Guangzhou. The total concentrations of the eight target MNPs in the PM2.5 samples were 100-990 ng/m3 (median 277 ng/m3) and the dominant MNPs were polyethylene, polyethylene terephthalate, and polyvinyl chloride, which contributed > 90 % of the MNPs. The new method allows the robust and accurate quantification of MNPs in atmospheric fine particles and will be useful in future studies on the environmental behaviors of MNPs and risks they pose.

Uncertainty quantification and propagation in atomistic machine learning

Abstract Machine learning (ML) offers promising new approaches to tackle complex problems and has been increasingly adopted in chemical and materials sciences. In general, ML models employ generic mathematical functions and attempt to learn essential physics and chemistry from large amounts of data. The reliability of predictions, however, is often not guaranteed, particularly for out-of-distribution data, due to the limited physical or chemical principles in the functional form. Therefore, it is critical to quantify the uncertainty in ML predictions and understand its propagation to downstream chemical and materials applications. This review examines existing uncertainty quantification (UQ) and uncertainty propagation (UP) methods for atomistic ML under the framework of probabilistic modeling. We first categorize the UQ methods and explain the similarities and differences among them. Following this, performance metrics for evaluating their accuracy, precision, calibration, and efficiency are presented, along with techniques for recalibration. These metrics are then applied to survey existing UQ benchmark studies that use molecular and materials datasets. Furthermore, we discuss UP methods to propagate uncertainty in widely used materials and chemical simulation techniques, such as molecular dynamics and microkinetic modeling. We conclude with remarks on the challenges and opportunities of UQ and UP in atomistic ML.

Influence of Air Quality Model Parameters on Pollution Concentration

Air pollution poses a significant threat to public health, ecosystems, and the global climate. Accurate prediction and effective management of air quality are of paramount importance, which, in turn, rely on sophisticated air quality models. These models integrate a variety of atmospheric parameters to simulate the dispersion of pollutants in the complex urban atmosphere, yet the influence of specific input parameters on predicted pollution concentrations has not been fully elucidated. This comprehensive study assesses how variations in model input parameters can lead to divergent pollution concentration outputs, with the goal of identifying those that are most critical to model accuracy. Using observational data from air quality monitoring stations in conjunction with meteorological records, the study explores the sensitivity of forecasted pollutant concentrations to fluctuations in model inputs such as emission source strength, atmospheric stability, wind speed and direction, diurnal heating patterns, chemical reaction rates, and boundary layer dynamics. Dispersion models are evaluated across different spatial and temporal scales to gauge their response to environmental variables and topographic features. The performance of these models is also assessed against satellite-derived pollutant measurements to encompass a broader geographical context. Through the application of numerical simulations and statistical analyses, the study quantifies the relative impact of each parameter. Cross-validation techniques, along with uncertainty quantification methods, are applied to ensure the reliability of the conclusions drawn. The research also incorporates the use of machine learning tools to identify complex patterns in the environmental data that may be missed by traditional modeling approaches. The abstract concludes that a detailed understanding of influential model parameters is essential for refining air quality predictions. Improvements in the accuracy of dispersion models will enable policymakers and urban planners to make better-informed decisions regarding air pollution control and mitigation strategies. This work forms the foundation for future advancements in the field of atmospheric sciences and encourages continued exploration into the interaction between anthropogenic activities, meteorological phenomena, and air quality outcomes

Toward spatio-temporal models to support national-scale forest carbon monitoring and reporting

Abstract National forest inventory (NFI) programs provide vital information on forest parameters’ status, trend, and change. Most NFI designs and estimation methods are tailored to estimate status over large areas but are not well suited to estimate trend and change, especially over small spatial areas and/or over short time periods (e.g. annual estimates). Fine-scale space-time indexed estimates are critical to a variety of environmental, ecological, and economic monitoring efforts. In the United States, for example, NFI data are used to estimate forest carbon status, trend, and change to support national, state, and local user group needs. Increasingly, these users seek finer spatial and temporal scale estimates to evaluate existing land use policies and management practices, and inform future activities. Here we propose a spatio-temporal Bayesian small area estimation modeling framework that delivers statistically valid estimates with complete uncertainty quantification for status, trend, and change. The framework accommodates a variety of space and time dependency structures, and we detail model configurations for different settings. The proposed framework is used to quantify forest carbon dynamics at an annual county-level across a 14 year period for the contiguous United States. Also, using an analysis of simulated data, we compare the proposed framework with traditional NFI estimators and offer computationally efficient algorithms, software, and data to reproduce results for benchmarking.

Optimized Air Quality Index and Meteorological Predictions with Machine Learning and IoT

Air Quality Index (AQI) prediction and forecasting play pivotal roles in assessing and managing air pollution, contributing to public health and environmental sustainability. This paper provides a comprehensive review of recent advancements, methodologies, challenges, and future directions in AQI prediction and forecasting. Recent research has seen a surge in the development of machine learning, statistical, and hybrid models for AQI prediction. These models leverage various input data sources such as meteorological data, satellite imagery, and pollutant emissions data to enhance prediction accuracy. Furthermore, the integration of advanced techniques like deep learning and ensemble modeling has shown promising results in capturing complex nonlinear relationships and improving forecast precision. Challenges persist, including the need for real-time data integration, model interpretability, and addressing spatial and temporal variations in air quality. Additionally, the impact of emerging factors such as climate change and urbanization on AQI prediction requires further investigation. Future research directions focus on the development of hybrid models that integrate multiple data sources, including sensor networks and IoT devices, to improve spatial and temporal resolution. Moreover, there is a growing emphasis on the incorporation of uncertainty quantification techniques to provide probabilistic forecasts and enhance decision-making under uncertainty. In conclusion, this paper underscores the importance of AQI prediction and forecasting in addressing air pollution challenges and promoting public health. By advancing methodologies, addressing challenges, and exploring emerging research avenues, we can strive towards more accurate, reliable, and actionable AQI predictions for sustainable urban development and environmental stewardship.

K-eμlator: emulating clustering effects of the k-essence dark energy

Abstract We build an emulator based on the polynomial chaos expansion (PCE) technique to efficiently model the non-linear effects associated with the clustering of the k-essence dark energy in the effective field theory (EFT) framework. These effects can be described through a modification of Poisson’s equation, denoted by the function μ(k, z), which in general depends on wavenumber k and redshift z. To emulate this function, we perform 200 high-resolution N-body simulations sampled from a seven-dimensional parameter space with the Latin hypercube method. These simulations are executed using the k-evolution code on a fixed mesh, containing 12003 dark matter particles within a box size of 400 Mpc/h. The emulation process has been carried out within UQLab, a MATLAB-based software specifically dedicated to emulation and uncertainty quantification tasks. Apart from its role in emulation, the PCE method also facilitates the measurement of Sobol indices, enabling us to assess the relative impact of each cosmological parameter on the μ function. Our results show that the PCE-based emulator efficiently and accurately reflects the behavior of the k-essence dark energy for the cosmological parameter space defined by $w_0 c_s^2 \text{CDM} +\sum m_{\nu }$. Compared against actual simulations, the emulator achieves sub-percent accuracy up to the wavenumber k ≈ 9.4 hMpc−1 for redshifts z ≤ 3. Our emulator provides an efficient and reliable tool for Markov chain Monte Carlo (MCMC) analysis, and its capability to closely mimic the properties of the k-essence dark energy makes it a crucial component in Bayesian parameter estimations. The code is publicly available at https://github.com/anourizo/k-emulator.

Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification

Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification

A software tool for applying Bayes' theorem in medical diagnostics

BackgroundIn medical diagnostics, estimating post-test or posterior probabilities for disease, positive and negative predictive values, and their associated uncertainty is essential for patient care.ObjectiveThe aim of this work is to introduce a software tool developed in the Wolfram Language for the parametric estimation, visualization, and comparison of Bayesian diagnostic measures and their uncertainty.MethodsThis tool employs Bayes' theorem to estimate positive and negative predictive values and posterior probabilities for the presence and absence of a disease. It estimates their standard sampling, measurement, and combined uncertainty, as well as their confidence intervals, applying uncertainty propagation methods based on first-order Taylor series approximations. It employs normal, lognormal, and gamma distributions.ResultsThe software generates plots and tables of the estimates to support clinical decision-making. An illustrative case study using fasting plasma glucose data from the National Health and Nutrition Examination Survey (NHANES) demonstrates its application in diagnosing diabetes mellitus. The results highlight the significant impact of measurement uncertainty on Bayesian diagnostic measures, particularly on positive predictive value and posterior probabilities.ConclusionThe software tool enhances the estimation and facilitates the comparison of Bayesian diagnostic measures, which are critical for medical practice. It provides a framework for their uncertainty quantification and assists in understanding and applying Bayes' theorem in medical diagnostics.

Nuclear Data Uncertainty Propagation in the PWR MOX/UO2 Core Transient Benchmark

In light water reactor (LWR) transient analyses using deterministic simulation codes, best-estimate plus uncertainty (BEPU) approaches complement the results obtained from simulation codes with uncertainty quantification. In this study, a stochastic sampling–based uncertainty analysis function was developed under the framework of BEPU, enabling uncertainty quantification of important parameters in LWR steady-state and transient analyses using SIMULATE5 and SIMULATE5-K. In the uncertainty quantification of the dynamic behavior of nuclear reactors, the uncertainty knowledge of thermal-hydraulic parameters was highlighted; however, the nuclear data uncertainty propagation to dynamic behavior was insufficient. To bridge the knowledge gap, the uncertainties of interest parameters in the Organisation for Economic Co-operation and Development/Nuclear Energy Agency and the U.S. Nuclear Regulatory Commission PWR MOX/UO2 Core Transient Benchmark were quantified considering cross-section and kinetics data uncertainties. Consequently, the uncertainties of the maximum fuel enthalpy important for rod ejection transient analyses evaluated on the basis of the generalized extreme value distribution were equally contributed by cross-section and kinetics data uncertainties.

Uncertainty Quantification in Automated Detection of Vertebral Metastasis Using Ensemble Monte Carlo Dropout.

The accurate and early detection of vertebral metastases is crucial for improving patient outcomes. Although deep-learning models have shown potential in this area, their lack of prediction reliability and robustness limits their clinical utility. To address these challenges, we propose a novel technique called Ensemble Monte Carlo Dropout (EMCD) for uncertainty quantification (UQ), which combines the Monte Carlo dropout and deep ensembles. In this retrospective study, we analyzed 11,468 abdominal computed tomography images from 116 patients diagnosed with vertebral metastases and 957 images from 11 healthy controls. Uncertainty was quantified and visualized using single number, predictive probability interval, posterior distribution and uncertainty class activation maps to provide a detailed understanding of prediction confidence. The EMCD model demonstrated superior performance compared with traditional UQ methods, achieving an area under the receiver operating characteristic curve (AUC) of 0.93 and an expected calibration error of 0.09, indicating high predictive accuracy and reliability. In addition, the model exhibited strong performance in handling out-of-distribution data. When data retention was applied based on uncertainty values, the AUC of the model improved to 0.96, highlighting the potential of uncertainty-driven data selection to enhance performance. The EMCD model represents a significant advancement in the automated detection of vertebral metastases, providing superior diagnostic accuracy and introducing a robust UQ framework to aid clinicians in making informed decisions.

Uncertainty quantification in genetic algorithm-optimized artificial intelligence-based mineral prospectivity models: automated hyperparameter tuning for support vector machines and random forest

Uncertainty quantification in genetic algorithm-optimized artificial intelligence-based mineral prospectivity models: automated hyperparameter tuning for support vector machines and random forest

RBNN with cycle-based parameter updates and its application to RUL prediction

Abstract Bayesian neural networks (BNNs) combine Bayesian theory with deep learning, providing a probabilistic interpretation of deep learning models. Traditional BNNs typically assume that network’s parameters follow a standard Gaussian distribution, which may not accurately reflect the actual situation. To address this issue, this paper proposes a Recurrent Bayesian Neural Network (RBNN) that employs two trainable Gaussian distributions as priors. Furthermore, an alternating cycle model parameter updation algorithm is designed accordingly, which not only prevents overfitting to specific data during training but also enhances the robustness of model parameters in error approximation. Additionally, this RBNN module can be integrated with classical neural network architecture conveniently. By integrating the RBNN with a temporal convolutional network (TCN), the effectiveness of the proposed approach was validated using a dataset for remaining useful life (RUL) prediction of gear and bearing components in civil aircraft. Comparative studies show that the integrated model outperforms traditional methods in terms of RUL prediction and uncertainty quantification, offering superior accuracy and robustness, and the RMSE of the RBNN prediction results is reduced by 17.7% and score is increased by 27.5% compared to BNN.

Graph convolutional networks-based method for uncertainty quantification of building design loads

Graph convolutional networks-based method for uncertainty quantification of building design loads

Prediction of Steady and Unsteady Flow Quantities Using Multiscale Graph Neural Networks

Abstract Analysis, optimization, and uncertainty quantification of the aerodynamic behavior of turbomachinery components is a fundamental part of the current industrial design process and requires the extensive use of compute-intensive computational fluid dynamics (CFD) simulations. This paper explores the potential of graph neural networks as surrogate models to accelerate the design process, for example, in a multi-fidelity framework. Graph neural networks promise to provide good estimates of flow quantities while maintaining the geometric accuracy at a fraction of the computational effort of CFD. To assess the performance of such methods, a state-of-the-art graph neural network is applied to a turbomachinery setup of industry-relevant mesh size. In particular, a multiscale graph neural network is used to overcome the problems of large information distances when applying message-passing based graph-net blocks to large meshes. The training database consists of a space-filling design of experiment of 100 CFD solutions with different geometries. The first use case encompasses the prediction of flow quantities of the complete fluid domain with 2.5×106 mesh points. The second use case focuses on predicting a single scalar (e.g., pressure) on surface meshes with up to 30×103 mesh points. In both cases, the networks predict time-averaged and unsteady flow fields on unstructured meshes of variable point sizes for new geometries not present in the training set. The results demonstrate the proficiency of the approach in predicting time-averaged and unsteady flow quantities on surfaces as well as for full fluid domains for new geometries.

Design and Uncertainty Evaluation of a Calibration Setup for Turbine Blades Vibration Measurement.

Turbomachinery engines face significant failure risks due to the combination of thermal loads and high-amplitude vibrations in turbine and compressor blades. Accurate stress distribution measurements are critical for enhancing the performance and safety of these systems. Blade tip timing (BTT) has emerged as an advanced alternative to traditional measurement methods, capturing blade dynamics by detecting deviations in blade tip arrival times through sensors mounted on the stator casing. This research focuses on developing an analytical model to quantify the uncertainty budget involved in designing a calibration setup for BTT systems, ensuring targeted performance levels. Unlike existing approaches, the proposed model integrates both operational variability and sensor performance characteristics, providing a comprehensive framework for uncertainty quantification. The model incorporates various operating and measurement scenarios to create an accurate and reliable calibration tool for BTT systems. In the broader context, this advancement supports the use of BTT for qualification processes, ultimately extending the lifespan of turbomachinery through condition-based maintenance. This approach enhances performance validation and monitoring in power plants and aircraft engines, contributing to safer and more efficient operations.

Dislocation dynamics in Ni: Parameterizing dislocation trajectories from atomistic simulations

We present a methodology using Differential Evolution Monte Carlo (DE-MC) to fit the parameters of an equation of motion to molecular dynamics (MD) trajectories of an a2〈110〉{111} edge dislocation in pure face-centered cubic (FCC) Ni, which is used as a model system. The motion of the edge dislocation under shear was observed using MD simulations with a classical effective potential, and the dislocation position was extracted. The parameters of interest were the effective mass, drag coefficient, and force experienced by the dislocation. Using DE-MC within a Bayesian framework, we estimated the joint posterior distribution of the parameters from the accepted samples produced by the DE-MC algorithm. The obtained distribution was then used to propagate the parameter uncertainties through the model, the equation of motion, which was used to predict the dislocation positions and velocities at the simulation timesteps as well as the uncertainty in the predictions. The mean fit was found to match the MD dislocation position data with a root mean square error (RMSE) of 0.2nm. We demonstrate how parameter distributions determined using the proposed methodology can be applied to obtain a mobility law for the class of dislocations studied with quantified uncertainties. This work is part of a longer-term study aiming to develop a surrogate model for precipitation strengthening, specifically dislocation-precipitate interactions, in Ni-based superalloys with uncertainty quantification. Accordingly, we also discuss the limitations of the selected model within this context and the necessary model extensions to overcome them. Published by the American Physical Society 2024