Quantification Of Epistemic Uncertainty Research Articles

Scenario analysis of local storylines to represent uncertainty in complex human-water systems

Storylines are important in evaluating the uncertainty inherent in complex human-water systems. The interrelated nature of qualitative and quantitative scenarios can enhance our ability to address the uncertainty of integrated modelling of complex systems. This study proposes a transdisciplinary approach that integrates social and environmental sciences to characterize and comprehend uncertainty in the dynamic interactions of key factors affecting a human-water system. We introduce a framework for representing uncertainty through linguistic and epistemic uncertainty quantification using storyline narratives in the context of a regional integrated dynamic model. A systematic exploration of uncertainty space is performed using storytelling, fuzzy sets, and low discrepancy sequences sampling methods. Scenario analysis is applied to the generated uncertain ensemble of projections to discover predominant storylines of interest. As a representative case of a human-water system operating in a developing country, we examine the uncertainty effects of a variety of drivers of climatic and socio-economic changes on key agriculture and water-related sectors in Pakistan’s Rechna Doab region. The findings revealed soil salinity and crop yield indices were the most uncertain and showed significant variance across all developed storylines. The 95th percentile for soil salinity in year 2100 was estimated to be nearly 60 % higher than the baseline level (year 2020). There was, however, considerable overlap in different socio-economic scenarios at the local scale, indicating that change in socio-economic conditions could not fully offset climate-related uncertainty. Our analysis provides better quantification and a deeper understanding of the uncertainty in integrated assessment modelling of coupled human-water systems and the complex relationships between inputs and outcomes.

Joint Failure Probability of Dams Based on Probabilistic Flood Hazard Analysis

Probabilistic risk methods are becoming increasingly accepted as a means of carrying out risk-informed decision making regarding the design and operation policy of structures such as dams. Probabilistic risk calculations require the quantification of epistemic and aleatory uncertainties not investigated through deterministic methodologies. In this hydrological study, a stochastic sampling methodology is employed to investigate the joint failure probability of three dams in adjacent similarly sized watersheds within the same hydrologic unit code (HUC) 6 basin. A probabilistic flood hazard analysis (PFHA) framework is used to simulate the hydrologic loading of a range of extreme precipitation events across the combined watershed area of the three studied dams. Precipitation events are characterized by three distinct storm types influential in the Tennessee Valley region with implications for weather variability and climate change. The stochastic framework allows for the simulation of hundreds of thousands of spillway outflows that are used to produce empirical bivariate exceedance probabilities for spillway discharge pairs at selected dams. System response curves that indicate the probability of failure given spillway discharge are referenced for each dam and applied to generate empirical bivariate failure probability (joint failure probability) estimates. The stochastic simulation results indicate the range of spillway discharges for each pair of dams that pose the greatest risk of joint failure. The estimate of joint failure considering the dependence of spillway discharges between dams is shown to be three to four orders of magnitude more likely (7.42 × 102 to 5.68 × 103) than estimates that assume coincident failures are the result of independent hydrologic events.

Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems

Machine learning methods for the construction of data-driven reduced order model models are used in an increasing variety of engineering domains, especially as a supplement to expensive computational fluid dynamics for design problems. An important check on the reliability of surrogate models is Uncertainty Quantification (UQ), a self assessed estimate of the model error. Accurate UQ allows for cost savings by reducing both the required size of training data sets and the required safety factors, while poor UQ prevents users from confidently relying on model predictions. We introduce two quality measures for the quantification of uncertainty that are suitable for high dimensional problems: the distribution of normalized residuals on validation data and the distribution of estimated uncertainties. While the distribution of estimated uncertainties evaluates how confident the model is in making predictions, the distribution of normalized residuals evaluates how accurate these uncertainty estimates are We examine several machine learning techniques, including both Gaussian processes and a family UQ-augmented neural networks: Ensemble neural networks (ENN), Bayesian neural networks (BNN), Dropout neural networks (D-NN), and Gaussian neural networks (G-NN). We evaluate UQ accuracy (distinct from model accuracy) We apply these metrics to two model data sets representative of complex dynamical systems: an ocean engineering problem in which a ship traverses irregular wave episodes, and a dispersive wave turbulence system with extreme events, the Majda–McLaughlin–Tabak model. We present conclusions concerning model architecture and hyperparameter tuning for use in applications of UQ to dynamical systems, such as active learning schemes.

Autobalanced Multitask Node Embedding Framework for Intelligent Education.

Recently, online education has become popular. Many e-learning platforms have been launched with various intelligent services aimed at improving the learning efficiency and effectiveness of learners. Graphs are used to describe the pairwise relations between entities, and the node embedding technique is the foundation of many intelligent services, which have received increasing attention from researchers. However, the graph in the intelligent education scenario has three noteworthy properties, namely, heterogeneity, evolution, and lopsidedness, which makes it challenging to implement ecumenical node embedding methods on it. In this article, an autobalanced multitask node embedding model is proposed, named MNE, and applied to the interaction graph, settling a few actual tasks in intelligent education. More specifically, MNE builds two purpose-built self-supervised node embedding learning tasks for heterogeneous evolutive graphs. Edge-specific reconstruction tasks are built according to the semantic information and properties of the heterogeneous edges, and an evolutive weight regression task is designed, aiding the model to perceive the evolution of learners' implicit cognitive states. Then, both aleatoric and epistemic uncertainty quantification techniques are introduced, achieving both task-and node-level weight estimation and instructing subtask autobalancing. Experimental results on real-world datasets indicate that the proposed model outperforms the state-of-the-art graph embedding methods on two assessment tasks and demonstrates the validity of the proposed multitask framework and subtask balancing mechanism. Our implementations are available at https://github.com/ccnu-mathits/MNE4HEN.

A simple and flexible bootstrap-based framework to quantify epistemic uncertainty of ground motion models by light gradient boosting machine

Ground motion models (GMM) incorporating proper epistemic uncertainty quantification are fundamental for seismic hazard analysis, structural seismic design, and earthquake risk management. However, this critical uncertainty has not been systematically addressed in recently prevalent machine learning (ML)-based GMMs. To rectify this gap, a straightforward and flexible framework based on bootstrapping is proposed for generic ML algorithms. Specifically, a group of datasets are generated through bootstrap resampling, each employed to develop an individual GMM. In the inference phase, the epistemic uncertainty is represented by the prediction samples from this ensemble of models. In case study, the Light Gradient Boosting Machine (LGBM), is applied to establish a GMM on the NGA-West2 database. Overfitting is mitigated by early-stopping, and its implications on uncertainty components and random-effect variability are thoroughly examined. Compared to artificial neural network (ANN) in literature, our work demonstrates a decrease in the standard deviation of inter- and intra-event variability by an average of 0.051 and 0.140, respectively. Moreover, our model adeptly captures the heteroscedasticity of epistemic uncertainty, particularly over different regions and within the range of the metadata with limited data. The significance of epistemic uncertainty is highlighted by its potential to diminish the statistical significance of seismic scaling effects at short-period pseudo spectral acceleration (PSA) for great events and at rock sites.

Epistemic and aleatoric uncertainty quantification for crack detection using a Bayesian Boundary Aware Convolutional Network

Epistemic and aleatoric uncertainty quantification for crack detection using a Bayesian Boundary Aware Convolutional Network

Uncertainty estimation for deep neural networks to improve the assessment of plumage conditions of chickens

The combination of computer vision with deep learning has become a popular tool for automation of labor-intensive monitoring tasks in modern livestock farming. However, uncontrolled and varying environmental conditions, which usually prevail in farmhouses, influence the performance of vision-based applications. Image quality can be reduced, for instance by occlusions, illumination or motions of the animals, which can influence the reliability of those applications. To address this issue, this study proposes an approach for the identification of uncertain neural-network predictions to improve the overall prediction quality. It proposes the direct quantification of aleatoric and epistemic uncertainty on the one hand and indirect estimation of uncertainty through the prediction of occlusions on the other hand. Our approach simultaneously integrates the different methods into an end-to-end trainable instance segmentation and regression model. The objective of this study was to first investigate how well the different measures can quantify the uncertainty of a prediction by comparing them to human uncertainty assessments. Then, it was analyzed whether the uncertainty estimations are capable to identify and reject erroneous predictions by evaluating the correlation between the predictive error and the uncertainty estimations. Finally, individual predictions were rejected based on the estimated uncertainties to analyze the effect on the overall accuracy. As a use-case, the developed methods were applied to the prediction of plumage conditions of chickens but also examined in a separate domain. The results showed that the outputs of our approaches for the estimation of aleatoric and epistemic uncertainty correlate to the predictive error of the model, and lead to increased performance when uncertain predictions are rejected. In contrast, the indirect method to identify occluded samples did not serve as a reliable indicator for uncertainty and could therefore not be used to improve the accuracy of the model outputs.

Belief reliability index determination method based on group decision‐making

AbstractFor newly developing systems, the determination of the reliability index is an important task in the system demonstration process that determines both the system design goal and the verification requirement. However, due to the insufficient data at the beginning of the system life cycle, the problem of epistemic uncertainty cannot be ignored. Researchers have proposed several belief reliability metrics and analysis methods to quantify the epistemic uncertainty of a system, but to the best knowledge of the authors, no relevant studies have been performed on system reliability index determination. In this paper, we propose a belief reliability index determination method based on group decision‐making. Using a similar system‐based reliability index determination method, this method comprehensively considers the demands of users and the capabilities of contractors. In addition, the Delphi method and Hadamard multiplication convex combination are combined to effectively solve the epistemic uncertainty quantification problem. The Delphi method is used to find the conflicts caused by multi‐expert scoring, and the Hadamard multiplication convex combination can integrate the scoring matrix of all experts, which makes the results more reasonable. Finally, a Low‐Earth‐orbit (LEO) satellite communication system is used to verify the effectiveness of our belief reliability index determination method.

EFFECT OF EPISTEMIC UNCERTAINTY ON SEISMIC FRAGILITY OF BASE-ISOLATED RC-BUILDING FRAME USING INCREMENTAL DYNAMIC ANALYSIS

The quantification and impact of epistemic uncertainty in the RC frame building can analyze due to recent advancements in processing power and the availability of reliable seismic vulnerability assessment techniques. In the present study, a base-isolated 10-story RC building frame is designed and analyzed per the latest Indian standards, with lead rubber bearing as the base isolation system. The selected base-isolated RC frame building was analyzed using probabilistic and deterministic approaches. As the fundamental epistemic uncertainty parameter, the mechanical strength of concrete and steel is considered. The Monte-Carlo method is determined the unpredictability of the RC frames' structures. The incremental dynamic analysis is conducted per the FEMA P695 guidelines to evaluate the wide range of 32 far-field ground motions on the base-isolated RC frame structure. Maximum roof drift ratio responses are utilized to generate the seismic fragility curve by adopting the several threshold values of damage index, such as, slight, moderate, severe, and complete. The traditional linear regression method is implemented to derive the seismic fragility curves of various damage thresholds. The study results reveal that the deterministic strategy is more likely to predict damage by 10 to 25 percent less than the probabilistic approach. The probabilistic analysis approach shows that epistemic uncertainty significantly affects the prediction of the seismic damage probability

Epistemic uncertainty quantification in structural systems using improved universal grey theory

It is important to account for uncertainties in structures during the analysis and design. Based on the source and nature, the uncertainties can be classified as aleatory and epistemic. Aleatory uncertainties arise due to the intrinsic randomness nature of physical system, whereas epistemic uncertainties realize on account of insufficient knowledge. When the information about the system is grey (i.e., partially available as range or interval), methods such as combinatorial approach, interval methods (IM) and universal grey theory (UGT) are generally adopted. The combinatorial optimization becomes computationally expensive when the dimension of uncertain system is large. Interval analysis leads to overestimation due to violations of the physical law and dependency problem. The satisfaction of the physical law (distributive law) that arises out of defining the arithmetic relations, contributes to the UGT free from dependency problem, and makes the approach more efficient. The traditional UGT is ineffective in certain conditions, wheneither one or both the bounds are negative with the absolute value of the upper bound being smaller i.e.,x¯⩽x̲. Therefore, this paper proposes a necessary modification in arithmetic operations to overcome the incapability of traditional UGT. The efficiency of proposed method is demonstrated through three numerical examples. Comparisons have been made with the conventional techniques to substantiate the proposed methodology, and the results obtained show that the proposed method is computationally efficient in terms of efforts and accuracy.

A Physics-Constrained Bayesian neural network for battery remaining useful life prediction

In order to predict the remaining useful life (RUL) of lithium-ion batteries, a capacity degradation model may be developed using either simplified physical laws or machine learning-based methods. It is observed that even though degradation models based on simplified physical laws are easy to implement, they may result in large error in the application of failure prognostics. While data-driven prognostics models can provide more accurate degradation forecasting, they may require a large volume of training data and may invoke predictions inconsistent with physical laws. It is also very challenging for existing methods to predict the RUL at the early stages of battery life. In this paper, we propose a Bayesian physics-constrained neural network for battery RUL prediction by overcoming limitations of the current methods. In the proposed method, a neural differential operator is learned from the first 100 cycles of data. The neural differential operator is modeled with a Bayesian neural network architecture that separates the fixed history dependence from the time dependence to isolate epistemic uncertainty quantification. Using the battery dataset presented in the paper by Severson et al. as an example, we compare our proposed method with a simplified physics-based degradation forecasting model and two data-driven prognostics models. The results show that the proposed physics-constrained neural network can provide more accurate RUL estimation than the other methods with the same group of training data. Most importantly, the proposed method allows for RUL prediction at earlier stages of the battery life cycle.

Bayesian neural networks with physics‐aware regularization for probabilistic travel time modeling

AbstractThe integration of data‐driven models such as neural networks for high‐consequence decision making has been largely hindered by their lack of predictive power away from training data and their inability to quantify uncertainties often prevalent in engineering applications. This article presents an ensembling method with function‐space regularization, which allows to integrate prior information about the function of interest, thus improving generalization performance, while enabling quantification of aleatory and epistemic uncertainties. This framework is applied to build a probabilistic ambulance travel time predictor, leveraging historical ambulance data provided by the Fire Department of New York City. Results show that the integration of a non‐Gaussian likelihood and prior information from a road network analysis yields appropriate probabilistic predictions of travel times, which could be further leveraged for emergency medical service (EMS) decision making.

Risk-targeted hazard for seismic design in New Zealand considering individual and societal risk targets

The promotion of risk-targeted hazard as the basis of seismic design internationally has increased over the past decade. Risk-targeted hazard estimates are derived through convolution of hazard curves with representative fragility functions and provide a means to target uniform risk across a region. Using risk targets also enables performance objectives of building codes that are consistent with other life risks and can include societal input into the expected performance of structures. Current design procedures using a uniform-hazard spectrum are unable to provide equal risk across the country due to variation in the shapes of hazard curves in different locations and uncertainty in structural response. In this article a framework is proposed, which can be used with national seismic hazard models, to produce risk-targeted hazard spectra for seismic design. This study extends the risk-targeted hazard framework through full quantification of epistemic uncertainty in seismic hazard and includes multiple risk targets for individual and societal risk at the building and city scales. The article describes a framework that can be used to adjust current seismic design force levels in New Zealand standards to target uniform seismic risk for buildings considering both the performance of individual buildings and the performance of cities.

Estimation of an imprecise power spectral density function with optimised bounds from scarce data for epistemic uncertainty quantification

In engineering and especially in stochastic dynamics, the modelling of environmental processes is indispensable in order to design structures safely or to determine the reliability of existing structures. Earthquakes or wind loads are examples of such environmental processes and can be described by stochastic processes. Such a process can be characterised by the power spectral density (PSD) function in the frequency domain. The PSD function determines the relevant frequencies and their amplitudes of a given time signal. For the reliable generation of a load model described by a PSD function, uncertainties that occur in time signals must be taken into account. This work mainly deals with the case where data is limited and it is infeasible to derive reliable statistics from the data. In such a case, it may be useful to identify bounds that characterise the data set. The proposed approach is to employ a radial basis function network to generate basis functions whose weights are optimised to obtain data-enclosing bounds. This results in an interval-based PSD function. No assumptions are required about the distribution of the data within those bounds. Thus, the spectral densities at each frequency are described by optimised bounds instead of relying on discrete values. The applicability of the imprecise PSD model is illustrated with recorded earthquake ground motions, demonstrating that it can be utilised for real world problems.

A STOCHASTIC DOMAIN DECOMPOSITION AND POST-PROCESSING ALGORITHM FOR EPISTEMIC UNCERTAINTY QUANTIFICATION

Partial differential equations (PDEs) are fundamental for theoretically describing numerous physical processes that are based on some input fields in spatial configurations. Understanding the physical process, in general, requires computational modeling of the PDE in bounded/unbounded regions. Uncertainty in the computational model manifests through lack of precise knowledge of the input field or configuration. Uncertainty quantification (UQ) in the output physical process is typically carried out by modeling the uncertainty using a random field, governed by an appropriate covariance function. This leads to solving high-dimensional stochastic counterparts of the PDE computational models. Such UQ-PDE models require a large number of simulations of the PDE in conjunction with samples in the high-dimensional probability space, with probability distribution associated with the covariance function. Those UQ computational models having explicit knowledge of the covariance function are known as aleatoric UQ (AUQ) models. The lack of such explicit knowledge leads to epistemic UQ (EUQ) models, which typically require solution of a large number of AUQ models. In this article, using a surrogate, post-processing, and domain decomposition framework with coarse stochastic solution adaptation, we develop an offline/online algorithm for efficiently simulating a class of EUQ-PDE models. We demonstrate the algorithm for celebrated bounded and unbounded spatial region models, with high-dimensional uncertainties.

An Uncertainty Quantification Framework for Counter Unmanned Aircraft Systems Using Deep Ensembles

The reliable detection and neutralization of unmanned aircraft systems (UASs), known as counter UAS (cUAS), is pivotal in restricted air spaces. The application of deep learning (DL) classifiers on electro-optical (EO) sensor data is promising for cUAS, but it introduces three key challenges. Specifically, DL-based cUAS produces point estimates at test time with no associated measure of uncertainty (softmax outputs produced by typical DL models are often overconfident predictions, resulting in unreliable measures of uncertainty), easily triggers false positive detections for birds and other aerial wildlife, and cannot accurately characterize out-of-distribution (OOD) input samples. In this work, we develop an epistemic uncertainty quantification (UQ) framework, which utilizes the advantages of DL while simultaneously producing uncertainty estimates on both in-distribution and OOD input samples. In this context, in-distribution samples refer to testing samples collected according to the same data generation process as the training data, and OOD samples refer to in-distribution samples that are intentionally perturbed in order to shift the distribution of the testing set away from the distribution of the training set. Our framework produces a distributive estimate of each prediction, which accurately expresses UQ, as opposed to a point estimate produced by standard DL. Through evaluation on a custom field-collected dataset consisting of images captured from EO sensors and in comparison to prior cUAS baselines, we show that our framework effectively expresses low and high uncertainty on in-distribution and OOD samples, respectively, while retaining accurate classification performance.

A unique physics-aided deep learning model for predicting viscosity of nanofluids

The viscosity of nanofluids can be influenced by many physical factors so it is difficult to obtain an accurate prediction model using only traditional theoretical model-driven methods or data-driven black-box models. This study proposes a modern approach named Physics Guided Deep Neural Network (PGDNN) for viscosity prediction that combines the data-driven models and physics-based theoretical models to achieve their complementary strengths and develop the modeling of physical processes. This PGDNN model is applied with a large number of data points (9000 data points) containing both experimental and simulated data of spherical nanoparticles Al2O3, CuO, SiO2, and TiO2. Further, this technique overcomes the overfitting issue and performs better than other traditional models while predicting unseen data. As far as we know, the PGDNN model is novel and not even used earlier to predict the viscosity of nanofluids. The learning performance of the proposed model is analyzed using different statistical performance indicators and Bayesian optimization is used for hyper-parameter tuning. Then, epistemic uncertainty quantification is performed to estimate the confidence level of the proposed model. Our PGDNN model outperformed various previous theoretical and computer-aided models with and RMSE = 0.0312. Moreover, a sensitivity analysis is performed and the results show that the volume fraction of particle is the most and viscosity of a base fluid is the second most significant parameters to determine the viscosity of nanofluids.

Iterative importance sampling with Markov chain Monte Carlo sampling in robust Bayesian analysis

Bayesian inference under a set of priors, called robust Bayesian analysis, allows for estimation of parameters within a model and quantification of epistemic uncertainty in quantities of interest by bounded (or imprecise) probability. Iterative importance sampling can be used to estimate bounds on the quantity of interest by optimizing over the set of priors. A method for iterative importance sampling when the robust Bayesian inference relies on Markov chain Monte Carlo (MCMC) sampling is proposed. To accommodate the MCMC sampling in iterative importance sampling, a new expression for the effective sample size of the importance sampling is derived, which accounts for the correlation in the MCMC samples. To illustrate the proposed method for robust Bayesian analysis, iterative importance sampling with MCMC sampling is applied to estimate the lower bound of the overall effect in a previously published meta-analysis with a random effects model. The performance of the method compared to a grid search method and under different degrees of prior-data conflict is also explored.

Sparse Gaussian Process Regression for Landslide Displacement Time-Series Forecasting

Landslide hazards are complex nonlinear systems with a highly dynamic nature. Accurate forecasting of landslide displacement and evolution is crucial for the prevention and mitigation of landslide hazards. In this study, a probabilistic landslide displacement forecasting model based on the quantification of epistemic uncertainty is proposed. In particular, the displacement forecasting problem is cast as a time-series regression problem with limited training samples and must be solved by statistical inference. The epistemic uncertainty of the landslide displacement series is depicted by the statistical properties of the function space constituted by the nonlinear mappings generated by the sparse Gaussian process regression. Data for our study was collected from the study area located in northwestern China. Other state-of-the-art probabilistic forecasting models have also been utilized for comparative analysis. The experimental results confirmed the superiority of the sparse Gaussian process in the modeling of landslide displacement series in terms of forecasting accuracy, uncertainty quantification, and robustness to overfitting.

Direct quantification of epistemic and aleatoric uncertainty in 3D U-net segmentation.

Purpose: To derive a multinomial probability function and quantitative measures of the data and epistemic uncertainty as direct output of a 3D U-Net segmentation network. Approach: A set of T1 brain MRI images were downloaded from the Connectome Project and segmented using FMRIB's FAST algorithm to be used as ground truth. A 3D U-Net neural network was trained with sample sizes of 200, 500, and 898 T1 brain images using a loss function defined as the negative logarithm of the likelihood based on a derivation of the definition of the multinomial probability function. From this definition, the epistemic and aleatoric uncertainty equations were derived and used to quantify maps of the uncertainty along with tissue segmentations. Results: Maps of the tissue segmentation along with the epistemic and aleatoric uncertainty, per voxel, are presented. The uncertainty decreased based on the increasing number of training data used to train the neural network. The neural network trained with 898 volumes resulted in uncertainty maps that were high primarily in the tissue boundary regions. The epistemic and aleatoric uncertainty were averaged over all test data (connectome and tumor separately), and the epistemic uncertainty showed a decreasing trend, as expected, with increasing numbers of data used to train the model. The aleatoric uncertainty showed a similar trend which was also expected as the aleatoric uncertainty is not expected to be as dependent on the number of training data. Conclusion: The derived uncertainty equations from a multinomial probability distribution were able to quantify the aleatoric and epistemic uncertainty per voxel and are applicable for all two-dimensional and three-dimensional neural networks.