Uncertainty In Model Estimates Research Articles

Uncertainty-Aware Health Diagnostics via Class-Balanced Evidential Deep Learning.

Uncertainty quantification is critical for ensuring the safety of deep learning-enabled health diagnostics, as it helps the model account for unknown factors and reduces the risk of misdiagnosis. However, existing uncertainty quantification studies often overlook the significant issue of class imbalance, which is common in medical data. In this paper, we propose a class-balanced evidential deep learning framework to achieve fair and reliable uncertainty estimates for health diagnostic models. This framework advances the state-of-the-art uncertainty quantification method of evidential deep learning with two novel mechanisms to address the challenges posed by class imbalance. Specifically, we introduce a pooling loss that enables the model to learn less biased evidence among classes and a learnable prior to regularize the posterior distribution that accounts for the quality of uncertainty estimates. Extensive experiments using benchmark data with varying degrees of imbalance and various naturally imbalanced health data demonstrate the effectiveness and superiority of our method. Our work pushes the envelope of uncertainty quantification from theoretical studies to realistic healthcare application scenarios. By enhancing uncertainty estimation for class-imbalanced data, we contribute to the development of more reliable and practical deep learning-enabled health diagnostic systems.

Enhancing Model Calibration and Uncertainty Estimation through Multi-Objective Optimization with SUFI-2 Algorithm

The research aimed to quantify the uncertainty in hydrological modeling results, especially in data-scarce areas like the Teesta River. The developed grid-based model from Texas Agricultural and Management University (TAMU) with the help of simulation plugin Quantum Soil and Water Assessment Tools (QSWAT) and calibration (CAL) & validation (VAL) using SWAT-Calibration Uncertainty Program (CUP). Multiple algorithms Minimize & Maximize (MIN & MAX), specifically with Sequential Uncertainty Fitting Version-2 (SUFI-2) algorithm (Best fitted simulation), were used as the popular for the many ideal simulations as possible that are within 95% of the true predictions. The research investigated how eleven different objective functions (OBJF) influenced the CAL results, parameterization, and surface water estimation for the stream flow of the Teesta River. While several OBJF performed well, eight of them, namely Chi-square (Chi2), Multiplicative (Multi), Modified Nash-Sutcliffe efficiency factor (MNSE), Nash-Sutcliffe efficiency (NSE), summation (SUM), Coefficient of determination (R2), Modified coefficient of determination (bR2), and Relative Squared Error (RSR), consistently yielded very similar results because of mostly used MIN methods. For instance, NSE resulted in a CAL & VAL score of 0.85 and 0.88, while R2, MNSE, and bR2 had scores of 0.88 and 0.89 during both CAL & VAL periods. All results from NSE, R2 and bR2 show a correlation between 0.84 to 0.90. However, other OBJF provided slightly different sensitivity values. The Percentage Bias (PBIAS) scored 0.76 for CAL and 0.77 for VAL. The Kling-Gupta efficiency (KGE) scored 0.79 during both the CAL & VAL periods. Additionally, the sum of squares (SSQR) obtained shallow values for both CAL & VAL processes. The aforementioned research shows that varied OBJF can affect CAL findings and hydrological modelling parameter values, emphasizing the need to use several OBJF in uncertainty analysis for more accurate water resource estimation the calibrated model and its outputs aid hydrologists and water resource administrators in agricultural and water assessment. They also support research on climate change's effects on water supply and quality, droughts, and food. The unique idea and flexible techniques make them suitable for most countries.

Fixed-Time Control for a Flexible Smart Structure With Actuator Failure: A Broad Learning System Approach.

This article proposes an adaptive fault-tolerant control (AFTC) approach based on a fixed-time sliding mode for suppressing vibrations of an uncertain, stand-alone tall building-like structure (STABLS). The method incorporates adaptive improved radial basis function neural networks (RBFNNs) within the broad learning system (BLS) to estimate model uncertainty and uses an adaptive fixed-time sliding mode approach to mitigate the impact of actuator effectiveness failures. The key contribution of this article is its demonstration of theoretically and practically guaranteed fixed-time performance of the flexible structure against uncertainty and actuator effectiveness failures. Additionally, the method estimates the lower bound of actuator health when it is unknown. Simulation and experimental results confirm the efficacy of the proposed vibration suppression method.

Estimation of airship states and model uncertainties using nonlinear estimators

This Airships are lighter than air vehicles and due to their growing number of applications, they are becoming attractive for the research community. Most of the applications require an airship autonomous flight controller which needs an accurate model and state information. Usually, airship states are affected by noise and states information can be lost in the case of sensor's faults, while airship model is affected by model inaccuracies and model uncertainties. This paper presents the application of nonlinear and Bayesian estimators for estimating the states and model uncertainties of neutrally buoyant airship. It is considered that minimum sensor measurements are available, and data is corrupted with process and measurement noise. A novel lumped model uncertainty estimation approach is formulated where airship model is augmented with six extra state variables capturing the model uncertainty of the airship. The designed estimator estimates the airship model uncertainty along with its states. Nonlinear estimators, Extended Kalman Filter and Unscented Kalman Filter are designed for estimating airship attitude, linear velocities, angular velocities and model uncertainties. While Particle filter is designed for the estimation of airship attitude, linear velocities and angular velocities. Simulations have been performed using nonlinear 6-DOF simulation model of experimental airship for assessing the estimator performances. 1−𝜎 uncertainty bound and error analysis have been performed for the validation. A comparative study of the estimator's performances is also carried out.

Robust synchronization of four-dimensional chaotic finance systems with unknown parametric uncertainties

ABSTRACT The inherent randomness in economic factors causes complex and irregular behaviour that affects financial system stability and economic growth. Such chaotic behaviour can make it difficult to synchronize financial systems. The chaotic finance system synchronization precision maintains financial stability and economic growth. In this paper, the controller design procedure assumes that the financial system is exposed to unknown bounded exogenous disturbances and model uncertainties. This research proposes a novel direct adaptive control strategy that achieves robust synchronization of two identical four-dimensional finance chaotic (FDFC) systems. The proposed controller establishes a faster, smoother synchronization error vector convergence to zero. The controller design procedure does not eliminate the closed-loop's nonlinear terms and is independent of the financial system parameters. These controller's attributes accomplish the closed-loop robust performance. Further, this controller uses real-time estimates of unknown model uncertainties and bounds to compensate for unknown exogenous disturbances. Computer simulation results and proofs of theoretical analysis based on the Lyapunov stability theory confirm that the proposed control technique compels the error vector trajectories to the origin in a short transient time with less active oscillations for all signals. The paper includes comparative computer simulations for verifying the theoretical findings.

Uncertainty-driven trustworthy defect detection for high-resolution powder bed images in selective laser melting

Selective laser melting (SLM) is known as one of the most promising metal additive manufacturing technologies, and how to ensure its consistent quality is still a main challenge, which urgently needs to be addressed. The metal powder spreading quality, as the first stage of SLM, has a direct impact on the subsequent forming and it is necessary to be monitored. However, existing deep learning-based methods for monitoring powder spreading quality often suffer from the problem of unreliability. The high-resolution property of powder bed images is often ignored, and the predicted results are not well evaluated. To address the above issues and achieve trustworthy defect detection, this paper proposes an uncertainty-driven trustworthy defect detection method for high-resolution powder bed images in SLM. A super resolution module based on ISDNet is first proposed to refine the upsampling process. Checkpoint ensemble is adopted for better achieving model uncertainty estimation only requiring a single training process, feature reuse helps reduce the computational load, and temperature scaling is used to further calibrate each ensemble member to get a more precise uncertainty. Besides, we design an uncertainty-driven model improvement method to further improve defect detection performance for regions with high uncertainty. Experiments demonstrate the effectiveness of our proposed method, and our work achieves a trustworthy defect detection for high-resolution powder bed images in SLM.

Degradation prediction of proton exchange membrane fuel cells with model uncertainty quantification

This study presents a new approach for state of health (SOH) estimation and degradation trend prediction of proton exchange membrane fuel cells (PEMFCs) considering model uncertainty quantification. The model uncertainty is considered as completely unknown in this paper, and the Bayesian framework-based model uncertainty and state simultaneous estimation (Bayes-MUSE) method is proposed to quantify the model uncertainty from the perspective of the form of its effect on the system state transfer, and this method of quantifying model uncertainty is first applied to PEMFC long-term aging prediction and RUL estimation. The proposed Bayes-MUSE method is compared with extended Kalman filter (EKF), Unscented Kalman filter (UKF) and Unscented Particle filter (UPF) methods in terms of SOH estimation, noise immunity, aging prediction performance and RUL estimation using two aging test datasets, and the method is verified to have excellent noise immunity and the highest long-term aging prediction and RUL estimation accuracy. Compared with EKF, UKF and UPF methods, the proposed Bayes-MUSE method has a maximum improvement of 93.95% and 52.2% in RUL estimation accuracy of the two data sets under quasi-dynamic and static conditions, respectively.

Adaptive Cooperative Formation Control of Autonomous Underwater Vehicles Based on Event-Triggered Mechanism

This paper investigates the coordinated motion control problem of autonomous underwater vehicles (AUVs), and an adaptive cooperative formation control method based on event-triggered mechanism is designed. In the estimation of external disturbance and model uncertainty, adaptive parameters are adopted to represent the upper bound of the learning weight matrix, which reduces the computational pressure of the system. It is proved theoretically that the signals in the system are semi-global uniformly ultimately bounded and independent of the initial state of the system. An event-triggered control mechanism is introduced, and the system updates the control law only when the event is triggered. Simulation results show that the controller can achieve the expected formation shape and save the communication resources of the system.

Uncertainty-Estimation-Based Prescribed Performance Pressure Control for Train Electropneumatic Brake Systems

Fast and precise pressure control for an electropneumatic brake system is essential for ensuring the safe operation of trains. However, the nonlinearity and uncertainties of the system make controller design challenging. This paper proposes a prescribed performance control method integrating an extended state observer to address this issue. A thermodynamical model of the brake cylinder is first built based on the pneumatic characteristics of the braking system, considering multiple modes, coupling effects, and input saturation. Then, an extended state observer is designed to estimate model uncertainty due to temperature variation and disturbances and to achieve online compensation of the model. A feedback control law with a specified prescribed performance function is developed based on the updated thermodynamic model to guarantee the transient and steady-state performance of the pressure control. A parameter adaptive method is also utilized to handle input saturation. The observer’s bounded convergence and stability analysis of the closed-loop control system is given using the Lyapunov theory. Compared experimental results are provided to verify the effectiveness of the proposed method.

Characterizing Errors Using Satellite Metadata for Eco‐Hydrological Model Calibration

AbstractUnderstanding the origins of errors between model predictions and catchment observations is a critical element in hydrologic model calibration and uncertainty estimation. Difficulties arise because there are a variety of error sources but only one measure of the total residual error between model predictions and catchment observations. One promising approach is to collect extra information a priori to characterize the data error before calibration. We implement here a new model calibration strategy for an ecohydrological model, using the satellite metadata information as a means to inform the model priors, to decompose data error from total residual error. This approach, referred to as Bayesian ecohydrological error model (BEEM), is first examined in a synthetic setting to establish its validity, and then applied to three real catchments across Australia. Results show that (a) BEEM is valid in a synthetic setting, as it can perfectly ascertain the true underlying error; (b) in real catchments the model error is reduced when utilizing the observation error variance as added error contributing to total error variance, while the magnitude of total residual error is more robust when utilizing metadata about the data quality proportionality as the basis for assigning total error variance; (c) BEEM improves model calibration by estimating the model error appropriately and estimating the uncertainty interval more precisely. Overall, our work demonstrates a new approach to collect prior error information in satellite metadata and reveals the potential for fully utilizing metadata about error sources in uncertainty estimation.

A fuzzy robust two-stage unscented Kalman filter method for uncertainty and state of charge estimation of lithium-ion batteries

For the practical state of charge (SoC) estimation of lithium-ion batteries, the unknown battery model uncertainties should be considered. Conventionally, the robust SoC estimation method uses a bounded matrix to model battery uncertainties. However, since the upper bound of the uncertainty is unknown, the robust estimation would be conservative and inaccurate. To accurately estimate the SoC of lithium-ion batteries, this paper proposes a novel fuzzy robust two-stage unscented Kalman filter (FRTSUKF) method to model the battery uncertainty with unknown statistical characteristics. The proposed estimator is able to estimate the model uncertainties and does not require the statistical characteristics of the uncertainties. Using the estimated uncertainties, the SoC estimation is corrected, and the destructive effect of the uncertainties on the estimation accuracy is eliminated correctly. To make the estimation more accurate and practical during experiments, the covariance matrix of the measurement noise is recursively adjusted using a fuzzy system. In addition, as the battery aging increases the model uncertainty, the proposed observer is able to adjust the SoC estimation at each battery life period using the estimated uncertainties. In the first stage, the state variables of the battery model are estimated. The covariance matrix of the measurement noise is recursively adjusted using a fuzzy system, which makes the estimation more accurate and practical during experiments. In the second stage, the unknown model uncertainties are estimated. Based on the estimated uncertainties, state variables, including SoC, are adjusted to compensate for the adverse effects of the model inaccuracy on the state estimation. The experimental results show that in addition to the ability for the battery model uncertainty estimation, the proposed estimator is more accurate, as much as 1 % and more robust than other Kalman-based methods for aged batteries with more uncertain models.

Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network.

Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use.

Path Following and Collision Avoidance of a Ribbon-Fin Propelled Underwater Biomimetic Vehicle-Manipulator System.

This paper addresses the problem of path following and dynamic obstacle avoidance for an underwater biomimetic vehicle-manipulator system (UBVMS). Firstly, the general kinematic and dynamic models of underwater vehicles are presented; then, a nonlinear model predictive control (NMPC) scheme is employed to track a reference path and collision avoidance simultaneously. Moreover, to minimize the tracking error and for a higher degree of robustness, improved extended state observers are used to estimate model uncertainties and disturbances to be fed into the NMPC prediction model. On top of this, the proposed method also considers the uncertainty of the state estimator, while combining a priori estimation of the Kalman filter to reasonably predict the position of dynamic obstacles during short periods. Finally, simulations and experimental results are carried out to assess the validity of the proposed method in case of disturbances and constraints.

Semi-Supervised Medical Image Segmentation With Voxel Stability and Reliability Constraints.

Semi-supervised learning is becoming an effective solution in medical image segmentation because annotations are costly and tedious to acquire. Methods based on the teacher-student model use consistency regularization and uncertainty estimation and have shown good potential in dealing with limited annotated data. Nevertheless, the existing teacher-student model is seriously limited by the exponential moving average algorithm, which leads to the optimization trap. Moreover, the classic uncertainty estimation method calculates the global uncertainty for images but does not consider local region-level uncertainty, which is unsuitable for medical images with blurry regions. In this paper, the Voxel Stability and Reliability Constraint (VSRC) model is proposed to address these issues. Specifically, the Voxel Stability Constraint (VSC) strategy is introduced to optimize parameters and exchange effective knowledge between two independent initialized models, which can break through the performance bottleneck and avoid model collapse. Moreover, a new uncertainty estimation strategy, the Voxel Reliability Constraint (VRC), is proposed for use in our semi-supervised model to consider the uncertainty at the local region level. We further extend our model to auxiliary tasks and propose a task-level consistency regularization with uncertainty estimation. Extensive experiments on two 3D medical image datasets demonstrate that our method outperforms other state-of-the-art semi-supervised medical image segmentation methods under limited supervision. The source code and pre-trained models of this method are available at https://github.com/zyvcks/JBHI-VSRC.

A Decision‐Relevant Factor‐Fixing Framework: Application to Uncertainty Analysis of a High‐Dimensional Water Quality Model

AbstractFactor Fixing (FF) is a common method for reducing the number of model parameters to lower computational cost. FF typically starts with distinguishing the insensitive parameters from the sensitive and pursues uncertainty quantification (UQ) on the resulting reduced‐order model, fixing each insensitive parameter at a fixed value. There is a need, however, to expand such a common approach to consider the effects of decision choices in the FF‐UQ procedure on metrics of interest. Therefore, to guide the use of FF and increase confidence in the resulting dimension‐reduced model, we propose a new adaptive framework consisting of four principles: (a) re‐parameterize the model first to reduce obvious non‐identifiable parameter combinations, (b) focus on decision relevance especially with respect to errors in quantities of interest (QoI), (c) conduct adaptive evaluation and robustness assessment of errors in the QoI across FF choices as sample size increases, and (d) reconsider whether fixing is warranted. The framework is demonstrated on a spatially‐distributed water quality model. The error in estimates of QoI caused by FF can be estimated using a Polynomial Chaos Expansion (PCE) surrogate model. Built with 70 model runs, the surrogate is computationally inexpensive to evaluate and can provide global sensitivity indices for free. For the selected catchment, just two factors may provide an acceptably accurate estimate of model uncertainty in the average annual load of Total Suspended Solids (TSS), suggesting that reducing the uncertainty in these two parameters is a priority for future work before undertaking further formal uncertainty quantification.

Revisiting Softmax for Uncertainty Approximation in Text Classification

Uncertainty approximation in text classification is an important area with applications in domain adaptation and interpretability. One of the most widely used uncertainty approximation methods is Monte Carlo (MC) dropout, which is computationally expensive as it requires multiple forward passes through the model. A cheaper alternative is to simply use a softmax based on a single forward pass without dropout to estimate model uncertainty. However, prior work has indicated that these predictions tend to be overconfident. In this paper, we perform a thorough empirical analysis of these methods on five datasets with two base neural architectures in order to identify the trade-offs between the two. We compare both softmax and an efficient version of MC dropout on their uncertainty approximations and downstream text classification performance, while weighing their runtime (cost) against performance (benefit). We find that, while MC dropout produces the best uncertainty approximations, using a simple softmax leads to competitive, and in some cases better, uncertainty estimation for text classification at a much lower computational cost, suggesting that softmax can in fact be a sufficient uncertainty estimate when computational resources are a concern.

Finite-time trajectory tracking control of dynamic positioning ship based on discrete-time sliding mode with decoupled sampling period

In this paper, the finite-time trajectory tracking control problem of a dynamic positioning ship with model uncertainty and time-varying environmental disturbances is investigated based on the discrete-time sliding mode with decoupled sampling period. To overcome the influence of the sampling period variation on the system’s control effect, a finite-time discrete sliding mode controller with decoupled sampling period is designed based on the new discrete-time adaptive fast power reaching law. The sampling period condition that guarantees the finite-time convergence of the system is given. A new finite-time high-order discrete disturbance observer is proposed to estimate model uncertainty and environmental disturbance, which reduces the dependence of the observer’s estimation accuracy on the sampling period. To realize the transformation from position tracking to velocity tracking in a finite time, a new finite-time discrete virtual velocity controller is designed. The proposed control strategy decouples the sampling period and enables the finite-time trajectory tracking control of the dynamic positioning ship over a wider sampling period range. The stability analysis demonstrates that all tracking errors of the closed-loop system can converge to a small domain near zero in a finite time. Two simulation examples verify the effectiveness of the proposed control method.

Spatial and temporal variability in summertime dissolved carbon dioxide and methane in temperate ponds and shallow lakes

AbstractSmall waterbodies have potentially high greenhouse gas emissions relative to their small footprint on the landscape, although there is high uncertainty in model estimates. Scaling their carbon dioxide (CO2) and methane (CH4) exchange with the atmosphere remains challenging due to an incomplete understanding and characterization of spatial and temporal variability in CO2 and CH4. Here, we measured partial pressures of CO2 (pCO2) and CH4 (pCH4) across 30 ponds and shallow lakes during summer in temperate regions of Europe and North America. We sampled each waterbody in three locations at three times during the growing season, and tested which physical, chemical, and biological characteristics related to the means and variability of pCO2 and pCH4 in space and time. Summer means of pCO2 and pCH4 were inversely related to waterbody size and positively related to floating vegetative cover; pCO2 was also positively related to dissolved phosphorus. Temporal variability in partial pressure in both gases weas greater than spatial variability. Although sampling on a single date was likely to misestimate mean seasonal pCO2 by up to 26%, mean seasonal pCH4 could be misestimated by up to 64.5%. Shallower systems displayed the most temporal variability in pCH4 and waterbodies with more vegetation cover had lower temporal variability. Inland waters remain one of the most uncertain components of the global carbon budget; understanding spatial and temporal variability will ultimately help us to constrain our estimates and inform research priorities.

Assessing Ecosystem State Space Models: Identifiability and Estimation

Hierarchical probability models are being used more often than non-hierarchical deterministic process models in environmental prediction and forecasting, and Bayesian approaches to fitting such models are becoming increasingly popular. In particular, models describing ecosystem dynamics with multiple states that are autoregressive at each step in time can be treated as statistical state space models (SSMs). In this paper, we examine this subset of ecosystem models, embed a process-based ecosystem model into an SSM, and give closed form Gibbs sampling updates for latent states and process precision parameters when process and observation errors are normally distributed. Here, we use simulated data from an example model (DALECev) and study the effects changing the temporal resolution of observations on the states (observation data gaps), the temporal resolution of the state process (model time step), and the level of aggregation of observations on fluxes (measurements of transfer rates on the state process). We show that parameter estimates become unreliable as temporal gaps between observed state data increase. To improve parameter estimates, we introduce a method of tuning the time resolution of the latent states while still using higher-frequency driver information and show that this helps to improve estimates. Further, we show that data cloning is a suitable method for assessing parameter identifiability in this class of models. Overall, our study helps inform the application of state space models to ecological forecasting applications where (1) data are not available for all states and transfers at the operational time step for the ecosystem model and (2) process uncertainty estimation is desired.

Synthesizing forecasts to inform decision‐making and advance ecological theory

Synthesizing forecasts to inform decision‐making and advance ecological theory