Model Uncertainty Research Articles

The Propagation and Reduction of Uncertainty Left Unquantified by Confidence Intervals, P-values, Neural Network Predictions, Posterior Distributions, and Other Statistical Results

Abstract In the use of statistical models to analyze data, there is not only the uncertainty quantified by the models but also uncertainty about which models are adequate for some purpose, such as weighing the evidence for or against a hypothesis of scientific interest. This paper provides methods for propagating such unquantified uncertainty to the results under a unified framework of adequate model averaging. Specifically, the weight of each model used in the average is the probability that it is the most useful model. To allow for the case that none of the models considered would be useful, a catch-all model is included in the model average at a different level of the hierarchy. The catch-all model is the vacuous model in imprecise probability theory, the model that puts no restrictions on the probabilities of statements about the unknown values of interest. That enables defining the proportion of the uncertainty left unquantified by a model as the probability that it is inadequate in the sense of being less useful than the catch-all model. A lower bound for the proportion of unquantified uncertainty of the averaged model decreases as more models are added to the average.

Sparse Bayesian learning using TMB (Template Model Builder)

Sparse Bayesian Learning, and more specifically the Relevance Vector Machine (RVM), can be used in supervised learning for both classification and regression problems. Such methods are particularly useful when applied to big data in order to find a sparse (in weight space) representation of the model. This paper demonstrates that the Template Model Builder (TMB) is an accurate and flexible computational framework for implementation of sparse Bayesian learning methods.The user of TMB is only required to specify the joint likelihood of the weights and the data, while the Laplace approximation of the marginal likelihood is automatically evaluated to numerical precision. This approximation is in turn used to estimate hyperparameters by maximum marginal likelihood. In order to reduce the computational cost of the Laplace approximation we introduce the notion of an “active set” of weights, and we devise an algorithm for dynamically updating this set until convergence, similar to what is done in other RVM type methods. We implement two different methods using TMB; the RVM and the Probabilistic Feature Selection and Classification Vector Machine method, where the latter also performs feature selection. Experiments based on benchmark data show that our TMB implementation performs comparable to that of the original implementation, but at a lower implementation cost. TMB can also calculate model and prediction uncertainty, by including estimation uncertainty from both latent variables and the hyperparameters. In conclusion, we find that TMB is a flexible tool that facilitates implementation and prototyping of sparse Bayesian methods.

A robust design of time-varying internal model principle-based control for ultra-precision tracking in a direct-drive servo stage

This paper proposes a robust design of the time-varying internal model principle-based control (TV-IMPC) for tracking sophisticated references generated by linear time-varying (LTV) autonomous systems. The existing TV-IMPC design usually requires a complete knowledge of the plant I/O (input/output) model, leading to the lack of structural robustness. To tackle this issue, we, in this paper, design a gray-box extended state observer (ESO) to estimate and compensate unknown model uncertainties and external disturbances. By means of the ESO feedback, the plant model is kept as nominal, and hence the structural robustness is achieved for the time-varying internal model. It is shown that the proposed design has bounded ESO estimation errors, which can be further adjusted by modifying the corresponding control gains. To stabilize the ESO-based TV-IMPC, a time-varying stabilizer is developed by employing Linear Matrix Inequalities (LMIs). Extensive simulation and experimental studies are conducted on a direct-drive servo stage to validate the proposed robust TV-IMPC with ultra-precision tracking performance (∼60nm RMSE out of ±80mm stroke).

Methodology of deployment of dependable FPGA based artificial intelligence as a service

The subject of study in this article is the models, methods, and principles of organization of entire lifecycle of Artificial Intelligence (AI) as a Service implemented with the use of Field Programmable Gate Array (FPGA). The purpose of this work is the improvement of the methodology of the deployment of dependable FPGA-based Artificial Intelligence as a Service by creating a complex of mutually agreed concept, principles, models, and methods considering the specifics of the use of heterogeneous computations of Artificial Intelligence and the possibility of realizing the unified protection of FPGA implementations. Tasks: to clarify the taxonomy of the dependability term within the proposed methodology; to propose the concept of deployment of dependable FPGA-based heterogeneous computations of Artificial Intelligence as a Service; to formulate the principle of tracing changes in FPGA projects and integrated environments during the entire lifecycle; to formulate the principle of unification of protection of FPGA implementations of heterogeneous computing of Artificial Intelligence as a Service; to formulate the principle of the product-service assessment of the availability of FPGA as a Service; and to discuss the promising directions of heterogeneous computations of AI. According to the tasks, the following results were obtained. The existing concepts of dependable systems deployment are discussed. The concept of the deployment of computations of Artificial Intelligence as a Service, which is obtained based on the improvement of paradigms of the creation and deployment of dependable systems and services, is proposed. The principle of tracing of changes, which assumes the updating of requirements during the lifecycle of FPGA projects, is proposed. The principle of the unification of protection, which combines and joins the consideration of various unique features of the FPGA instance to protect the implementation and the set of cyberthreats for the service as a whole, is proposed. The principle of the product-service assessment, which considers parameters and indicators of availability, is proposed. The perspective of the progress of non-electronic mediums for heterogeneous computations with the use of a photonic implementations of Artificial Intelligence computations to ensure improved performance and reduced energy consumption is discussed. Conclusions. One of the main contributions of this research is that in the proposed methodology, the set of principles, models, and methods of deployment of Artificial Intelligence as a Service under conditions of changing requirements and integrated environments, and the need for mechanism of licensing protection of each instance of the system are developed, which allows to reduce model uncertainty by considering various stages of the lifecycle of dependable FPGA implementation using heterogeneous computations.

Nonlinear dynamic inversion based full envelope robust flight control for coaxial compound helicopter

The flight control system of coaxial compound helicopters (CCHs) is a complex multi-variable system characterized by redundant control effectors, strong nonlinear characteristics, and multiple flight modes. This paper presents a unified robust control framework tailored for CCH full-envelope flight, utilizing nonlinear dynamic inversion and extended state observer techniques. The framework is specifically designed to tackle issues arising from a multi-mode nature, parametric uncertainties, and external disturbances. Within the unified framework, the study addresses inherent complexities such as nonlinearities and cross-coupling effects in CCH by leveraging nonlinear dynamic inversion to formulate flight control laws for both inner and outer-loops. To mitigate model uncertainties and external disturbances, the extended state observer is employed to estimate lumped disturbance effectively. Control allocation for both inner and outer-loops is devised to adapt to changes in control effector authorities and flight modes. Furthermore, an exponential control allocation strategy is proposed for the outer-loop to ensure a smooth pitch angle during transition flight. The effectiveness of the proposed control laws and control allocation strategies in achieving precise attitude tracking, transition flight and mission task elements (MTEs), even in the presence of notable model uncertainties and external disturbances, is demonstrated through numerical simulations.

Neutrinoless double beta decay in the minimal type-I seesaw model: mass-dependent nuclear matrix element, current limits and future sensitivities

In this work we discuss the neutrino mass dependent nuclear matrix element (NME) of the neutrinoless double beta decay process and derive the limit on the parameter space of the minimal Type-I seesaw model from the current available experimental data as well as the future sensitivities from the next-generation experiments. Both the explicit many-body calculations and naive extrapolations of the mass dependent NME are employed in the current work. The uncertainties of the theoretical nuclear structure models are taken into account. By combining the latest experimental data from 76Ge-based experiments, GERDA and MAJORANA, the 130Te-based experiment, CUORE and the 136Xe-based experiments, KamLAND-Zen and EXO-200, the bounds on the parameter space of the minimal Type-I seesaw model are obtained and compared with the limits from other experimental probes. Sensitivities for future experiments utilizing 76Ge-based (LEGEND-1000), 82Se-based (SuperNEMO), 130Te based (SNO+II) and 136Xe-based (nEXO), with a ten-year exposure, are also derived.

Spatiotemporally Detailed Quantification of Air Quality Benefits of Emissions-Part II: Sensitivity to Study Parameters and Assumptions.

Adjoint modeling, using U.S. EPA's Community Multiscale Air Quality (CMAQ), has been performed to provide location-specific monetized health benefits from the controls of primary PM2.5 and PM2.5 precursors (NO x , SO2, and NH3) across North America. Source-to-health benefit relationships are quantified using a benefit-per-ton (BPT) metric, accounting for the impacts on premature mortality due to long-term exposure to fine particulate matter. In the base analysis, the approach used a 12 km resolution, four 2-week episodes chosen to capture annual responses, emissions for 2016, and the Global Exposure Mortality Model (GEMM) to link exposures to premature mortality. Here, we investigate the impacts those choices have on results using a range of sensitivity analyses. The choice of four representative episodes led to relatively little bias and error. Finer model resolution, investigated by comparing 36, 12, 4, and 1 km simulations over two urban areas, tended to increase BPT estimates, though the impact was inconsistent between different regions. While BPTs and burden estimates were consistent across resolutions over New York City, they sharply increased for Los Angeles, particularly for NOx and ammonia, leading to 90% increase in burden estimates at 1 km resolution. We find that, for primary PM2.5 emissions, better resolved population distribution is the main contributing factor to higher BPTs, but for secondary precursor emissions (ammonia and NOx), higher model resolution that avoids dilution in coarser grids is more important. Changing emissions from 2016 to 2001 and 2028 resulted in fairly consistent primary PM2.5 BPTs but impacted the BPTs for NOx and ammonia more significantly due to changes in SO2 emissions. We found that BPTs tend to stabilize, as emission changes in 2028 lead to a lower deviation from 2016 BPTs compared to changes from the 2001 episode. The role of the epidemiological model also led to relatively modest uncertainties, 15-30% depending on the species, even when different shapes of concentration-response functions were employed. We found the impact of the choice of CRF to be larger or comparable in size to the reported epidemiological model uncertainties for log-linear CRFs. The adjoining approach proved robust to modeling choices in providing BPT estimates that are highly granular across locations and emitted species.

A methodology for accelerated design of irradiation experiments: Demonstration via an OpenFOAM-informed TRANSURANUS simulation

The design and licensing of irradiation experiments typically require dedicated neutronics, thermo-hydraulics, and thermo-mechanical simulations, among which neutronics and thermo-hydraulics present considerable computational burdens. In this work, we propose a methodology to potentially reduce the required number of simulations, thus accelerating the design of irradiation experiments. The proposed methodology is based on the systematic comparison of intrinsic modelling uncertainties of relevant figures of merit with either the impact of the reciprocal influence of neutronics, thermo-hydraulics, and thermo-mechanics, or the impact of design parameters of the irradiation experiment itself. We exemplify the application of the proposed methodology on the design of irradiation experiments with varying americium content in the in-pile test section of the MYRRHA research fast reactor, demonstrating the possibility of reducing the number of required thermo-hydraulic and thermo-mechanic simulations.

Probabilistic Programming for Transportable Source Characterization and Uncertainty Quantification of the North Korean Nuclear Tests 2006–2017

Abstract We introduce a transportable technique to determine the yield and depth of burial (DOB) from seismic source spectra of underground nuclear explosions. We demonstrate this technique on the six declared North Korean nuclear tests. This approach derives source spectra in absolute units from regional phase (Pg) amplitudes by correcting the observations for geometric spreading, attenuation, and site amplification. We couple the source spectra and explosion source models with a probabilistic programming framework that integrates deep learning techniques and Bayesian modeling. This approach permits the exchange of information across various data categories to quantify both the data and model uncertainty. This technique stands out as an innovative use of broad-area propagation models, making it transportable across various geologic settings. This method proves to be effective in scenarios with diverse and/or limited observational data, even when the source depth is unknown. We present new independent estimates of absolute yield and DOB that are consistent with the prior assessments, underscoring the potential of this method in enhancing transportable nuclear explosion monitoring capabilities.

Time‐varying copula‐based compound flood risk assessment of extreme rainfall and high water level under a non‐stationary environment

AbstractQuantifying flood risk depends on accurate probability estimation, which is challenging due to non‐stationarity and the combined effects of multiple factors in a changing environment. The threat of compound flood risks may spread from coastal areas to inland basins, which have received less attention. In this study, a framework based on time‐varying copulas was introduced for the treatment of compound flood risk and bivariate design in non‐stationary environments. Archimedean copulas were developed to diagnose the non‐stationary trends of flood risk. Return periods, average annual reliabilities, and bivariate designs were estimated. Model uncertainty was analyzed by comparing the results for stationary and non‐stationary conditions. The case study investigated the extreme rainfall and water level series from the Qinhuai River Basin and the Yangtze River in China. The results showed that marginal distributions and correlations are non‐stationary in all bivariate combinations. Ignoring composite effects may lead to inappropriate quantification of flood risk. Excluding non‐stationarity may lead to risk over or underestimation. It showed the limitations of the 1‐day scale and quantified the uncertainty of non‐stationary models. This study provided a flood risk assessment framework in a changing environment and a risk‐based design technique, which is essential for climate change adaptation and water management.

Kinematic Source Properties of the 2023 Mw 7.7 Türkiye Earthquake Inferred from Near-Fault Strong Ground Motions

Abstract The 2023 Mw 7.7 Türkiye earthquake provided densely observed near-fault ground-motion data, which enabled us to investigate their characteristics and generations. This study aimed to obtain an intuitive understanding of the characteristics of near-fault strong ground motions and their relationship with kinematic source properties using a simple source model. Synthetic ground motions were calculated using the discrete wave number method and compared with observed ground motions. The observed strong motions were used after a time correction based on the first P-wave arrival. Results showed that near-fault pulse-like velocities (0.02–0.6 Hz) were generally explained by the simple kinematic source model that assumed a continuous rupture propagation along the faults. Comparisons at individual sites indicate the various rupture properties of the fault segments. Near-fault ground motions along the Narli segment were explained by the fault plane along the surface trace and not by aftershock distribution. A supershear rupture propagation speed is necessary for a part of the Amanos segment. Rupture stops and restarts were implied near the stepover along the Amanos segment. Empirical site factors estimated by generalized inversion techniques did not show peaks around the frequency of observed large velocity pulses, indicating that the velocity pulses were mainly generated by the source. The model bias and uncertainties were also examined to supplement the simplified subsurface structure used in the simulation. Our results will help to associate the qualitative and quantitative aspects of the kinematic source process with near-fault strong ground motions.

Applying a 2D Variational Rigid Block Modeling Method to Rubble Stone Masonry Walls Considering Uncertainty in Material Properties

ABSTRACT The computational cost of detailed micro-modeling of masonry structures can be minimized with rigid block analysis methods, where a variational formulation is particularly interesting as it allows for nonlinear pushover analysis. This work investigates the accuracy of this modeling method in predicting the response of rubble stone masonry walls in shear compression tests. To this end, we extend the mathematical programming problem by including cohesion, compressive strength, and tensile strength in the formulation and implement an improved pushover procedure, which guards compatibility of the displaced shape and crack patterns at the intersection of elastic and rigid contact models. Additionally, we reduce model uncertainty, introduced through sampling material properties with distributions estimated based on results from tests on mortar masonry joints and on mortar specimens, by correlating the predicted crack patterns with the observed ones. The findings show that (i) model falsification based on crack pattern is an effective method for reducing the parameter uncertainty of stone masonry walls; and that (ii) the improved pushover procedure predicts the pre- and post-peak response of stone masonry walls very well with analysis times of less than 20 minutes for 2D models where stones and mortar are modeled explicitly.

Tectonic Landform and Lithologic Age Impact Uncertainties in Fault Displacement Hazard Models

AbstractTectonic landforms and surficial lithologic age are essential data for producing quality late Quaternary fault maps and predicting coseismic fault rupture location before an earthquake. However, we lack a clear understanding of the relationship between tectonic landforms and shallow earthquake processes and how lithologic age relates to landform preservation. We assess how fault location error (rupture‐to‐fault separation distance) and coseismic displacement residual (difference between observed and predicted coseismic displacement) vary with tectonic landform and lithologic age for four historical earthquakes. Certain tectonic landforms identified before these earthquakes correlate with lower fault location errors and median displacements below model predictions. Faults cutting Holocene units exhibit the largest location errors, reflecting surface processes that erode or bury fault evidence. This study shows that tectonic landforms and lithologic age have a significant impact on fault location uncertainty and coseismic displacement, which should be considered in fault mapping and fault displacement assessment.

Mapping aboveground biomass in Indonesian lowland forests using GEDI and hierarchical models

Spaceborne lidar (light detection and ranging) instruments such as the Global Ecosystem Dynamics Investigation (GEDI) provide a unique opportunity for global forest inventory by generating broad-scale measurements sensitive to the vertical arrangement of plant matter as a supplement to in situ measurements. Lidar measurables are not directly relatable to most physical attributes of interest, including biomass, and therefore must be related through statistical models. Further, GEDI observations are not spatially complete, necessitating methods to convert the incomplete samples to predictions of area averages/totals. Such methods can face challenges in equatorial and persistently cloudy areas, such as Indonesia, where the density of quality observations is diminished. We developed and implemented a hierarchical model to produce gap-free maps of aboveground biomass density (AGBD) at various resolutions within the lowlands of Jambi province, Indonesia. A biomass model was trained between local field plots and a metric from GEDI waveforms simulated with coincident airborne laser scanning (ALS) data. After selecting a locally suitable ground-finding algorithm setting, we trained an error model depicting the discrepancies between the simulated and GEDI-observed waveforms. Finally, a geostatistical model was used to model the spatial distribution of the on-orbit GEDI observations. These three models were nested into a single hierarchical model, relating the spatial distribution of GEDI observations to field-measured AGBD. The model allows spatially complete predictions at arbitrary resolutions while accounting for uncertainties at each stage of the relationship. The model uncertainties were low relative to the predicted biomass, with a median relative standard deviation of 8% at the 1 km resolution and 26% at the 100 m resolution. The spatially consistent information on AGBD provided by our model is beneficial in support of sustainable forest management, carbon sequestration initiatives and the mitigation of climate change. This is particularly relevant in a dynamic tropical landscape like Jambi, Indonesia in order to understand the impacts of land-use transformations from forests to cash crops like oil palm and rubber. More generally, we advocate for the use of hierarchical models as a framework to account for multiple stages of relationships between field and sensor data and to provide reliable uncertainty audits for final predictions.

An Adaptive Characteristic Model-Based Event-Triggered Sigmoid Prescribed Performance Control Approach for Tracking the Trajectory of Autonomous Underwater Vehicles

This paper introduces an event-triggered sigmoid prescribed performance control method, enhanced by an adaptive characteristic model, for tracking the trajectory of autonomous underwater vehicles (AUVs). The AUV model is simplified into a function reliant solely on second-order parameter information through the use of characteristic modeling and a compression algorithm, which is then approximated by a neural network. We propose integrating prescribed performance control into event-triggered sliding mode control to accelerate convergence in AUV trajectory tracking. A novel prescribed performance function is employed in this integration, creating an event-triggered, non-singular terminal sliding mode control strategy. The stability of this controller is rigorously proven. This control strategy is not only robust against model uncertainties but also mitigates the jitter commonly associated with sliding mode control and the singularities from preset performance control due to sudden random disturbances. Comparative simulation experiments demonstrate that the proposed control method achieves superior control accuracy and a quicker response.

Deep Koopman-operator-based model predictive control for free-floating space robots with disturbance observer

This paper investigates the optimal tracking control problem of free-floating space robots in the presence of non-ideal factors, such as model uncertainties and external disturbances. To address this issue, we first leverage the deep Koopman operator, facilitated with a deep neural network, to establish an offline formulation of a global linearization model for a micro-nano free-floating space robot. Based on the estimated linearization model, we then employ the model predictive control method for online optimization control, achieving a significantly reduced computational burden. Additionally, a decay factor is integrated into the model predictive control optimization objective function to balance the precision of joint angles tracking with the suppression of base satellite attitude disturbance. To further address the modeling inaccuracies and external disturbances, we incorporate a disturbance observer based on a radial basis function neural network for online compensation within the model predictive control framework. This augmentation enhances tracking precision and robustness. Several groups of simulation results are carried out to demonstrate the effectiveness of the proposed method, showing its capability of energy consumption, disturbance rejection and enhanced robustness. These highlight the potential of the proposed method to improve the control performance of free-floating space robots, even in the absence of a precise dynamic model.

RBFNN-based global fast terminal sliding mode control for fully controlled doubly fed induction generator

In this paper, a global fast terminal sliding mode control (GFTSMC) with radial basis function neural network (RBFNN) is proposed for fully controlled doubly fed induction generator (FC-DFIG) wind power system. Initially, to realize the maximum power point tracking (MPPT) of the FC-DFIG power generation system, a GFTSMC technique is introduced. Additionally, a new sliding mode reaching law is explored to enhance the respective speed and mitigate the sliding mode shaking amplitude. Subsequently, RBFNN is deployed to approximate external interferences, model uncertainties, and other unpredicted factors. The effectiveness and finiteness of the proposed control technique are affirmed through comprehensive simulation and experimental studies.

How Model Organisms and Model Uncertainty Impact Our Understanding of the Risk of Sublethal Impacts of Toxicants to Survival and Growth of Ecologically Relevant Species.

Understanding how sublethal impacts of toxicants affect population-relevant outcomes for organisms is challenging. We tested the hypotheses that the well-known sublethal impacts of methylmercury (MeHg) and a polychlorinated biphenyl (PCB126) would have meaningful impacts on cohort growth and survival in yellow perch (Perca flavescens) and Atlantic killifish (Fundulus heteroclitus) populations, that inclusion of model uncertainty is important for understanding the sublethal impacts of toxicants, and that a model organism (zebrafish Danio rerio) is an appropriate substitute for ecologically relevant species (yellow perch, killifish). Our simulations showed that MeHg did not have meaningful impacts on growth or survival in a simulated environment except to increase survival and growth in low mercury exposures in yellow perch and killifish. For PCB126, the high level of exposure resulted in lower survival for killifish only. Uncertainty analyses increased the variability and lowered average survival estimates across all species and toxicants, providing a more conservative estimate of risk. We demonstrate that using a model organism instead of the species of interest does not necessarily give the same results, suggesting that using zebrafish as a surrogate for yellow perch and killifish may not be appropriate for predicting contaminant impacts on larval cohort growth and survival in ecologically relevant species. Our analysis also reinforces the notion that uncertainty analyses are necessary in any modeling assessment of the impacts of toxicants on a population because it provides a more conservative, and arguably realistic, estimate of impact. Environ Toxicol Chem 2024;43:2122-2133. © 2024 SETAC.

A Conformal Prediction Approach to Enhance Predictive Accuracy and Confidence in Machine Learning Application in Chronic Diseases.

Heterogeneity in chronic malignancies raises an increasing interest for the integration and study of predictive models. This study presents a machine learning model approach to predict outcomes and improve their trustworthiness in multi-factorial diseases with highly heterogeneous outcomes, like Chronic Lymphocytic Leukemia (CLL). We incorporated Conformal Prediction to quantify our models uncertainty, and generate confident personalized prediction outcomes that can be integrated into clinical practice.

Highly Selective Novel Heme Oxygenase-1 Hits Found by DNA-Encoded Library Machine Learning beyond the DEL Chemical Space.

DNA-encoded library (DEL) technology, especially when combined with machine learning (ML), is a powerful method to discover novel inhibitors. DEL-ML can expand a larger chemical space and boost cost-effectiveness during hit finding. Heme oxygenase-1 (HO-1), a heme-degrading enzyme, is linked to diseases such as cancer and neurodegenerative disorders. The discovery of five series of new scaffold HO-1 hits is reported here, using a DEL-ML workflow, which emphasizes the model's uncertainty quantification and domain of applicability. This model exhibits a strong extrapolation ability, identifying new structures beyond the DEL chemical space. About 37% of predicted molecules showed a binding affinity of K D < 20 μM, with the strongest being 141 nM, amd 14 of those molecules displayed >100-fold selectivity for HO-1 over heme oxygenase-2 (HO-2). These molecules also showed structural novelty compared to existing HO-1 inhibitors. Docking simulations provided insights into possible selectivity rationale.