Validation Of Probabilistic Model Research Articles

Experimental and probabilistic model validation of ultrasonic MEMS transceiver for blood glucose sensing.

In contrast to traditional laboratory glucose monitoring, recent developments have focused on blood glucose self-monitoring and providing patients with a self-monitoring device. This paper proposes a system based on ultrasound principles for quantifying glucose levels in blood by conducting an in-vitro experiment with goat blood before human blood. The ultrasonic transceiver is powered by a frequency generator that operates at 40 kHz and 1.6 V, and variations in glucose level affect the ultrasonic transceiver readings. The RVM probabilistic model is used to determine the variation in glucose levels in a blood sample. Blood glucose levels are measured simultaneously using a commercial glucose metre for confirmation. The experimental data values proposed are highly correlated with commercial glucose metre readings. The proposed ultrasonic MEMS-based blood glucometer measures a glucose level of [Formula: see text] mg/dl. In the near future, the miniature version of the experimental model may be useful to human society.

Data-Driven Abstractions for Robots With Stochastic Dynamics

This article describes the construction of stochastic, data-based discrete abstractions for uncertain random processes continuous in time and space. Motivated by the fact that modeling processes often introduce errors which interfere with the implementation of control strategies, here the abstraction process proceeds in reverse: the methodology does not abstract models; rather it models abstractions. Specifically, it first formalizes a template for a family of stochastic abstractions, and then fits the parameters of that template to match the dynamics of the underlying process and ground the abstraction. The article also shows how the parameter-fitting approach can be implemented based on a probabilistic model validation approach which draws from randomized algorithms, and results in a discrete abstract model which is approximately simulated by the actual process physics, at a desired confidence level. In this way, the models afford the implementation of symbolic control plans with probabilistic guarantees at a desired level of fidelity.

Adaptive Variable-Fidelity Analysis and Design Using Dynamic Fidelity Indicators

The purpose of the current study is to develop a decision-making model of a dynamic fidelity indicator (DFI) for adaptive variable-fidelity (VF) analysis and to apply it to practical engineering problems, including computational design optimization. The biggest advantage is considerable computation efficiency while satisfying solution accuracy. Bayesian-based DFI and difference-based DFI are formulated as probabilistic model validation metrics to quantify model-form uncertainties attributed to the VF analysis. Two adaptive VF surrogate models of the VF kriging with regression and sample reinterpolation and the simple VF kriging with an additive bridge function are developed. The VF kriging models are integrated with the DFIs into an efficient global optimization design framework for the design problems, which was further improved by the multipoint and multi-objective infill sampling criteria. Validation of the developed framework is carried out with two analytic functions for solution optimality and computational efficiency. As a practical design application, the NLR 7301 multi-element airfoil with a flap is designed for high lift while the model-from uncertainty is defined by different flow solutions of the Euler and Reynolds-averaged Navier–Stokes equations. The results show that the design optimization becomes more efficient with great accuracy by the adaptive VF analysis with the DFIs.

Separation of aleatory and epistemic uncertainty in probabilistic model validation

This paper investigates model validation under a variety of different data scenarios and clarifies how different validation metrics may be appropriate for different scenarios. In the presence of multiple uncertainty sources, model validation metrics that compare the distributions of model prediction and observation are considered. Both ensemble validation and point-by-point approaches are discussed, and it is shown how applying the model reliability metric point-by-point enables the separation of contributions from aleatory and epistemic uncertainty sources. After individual validation assessments are made at different input conditions, it may be desirable to obtain an overall measure of model validity across the entire domain. This paper proposes an integration approach that assigns weights to the validation results according to the relevance of each validation test condition to the overall intended use of the model in prediction. Since uncertainty propagation for probabilistic validation is often unaffordable for complex computational models, surrogate models are often used; this paper proposes an approach to account for the additional uncertainty introduced in validation by the uncertain fit of the surrogate model. The proposed methods are demonstrated with a microelectromechanical system (MEMS) example.

Probabilistically valid stochastic extensions of deterministic models for systems with uncertainty

Models capable of capturing and reproducing the variability observed in experimental trials can be valuable for planning and control in the presence of uncertainty. This paper reports on a new data-driven methodology that extends deterministic models to a stochastic regime and offers probabilistic guarantees of model fidelity. From an acceptable deterministic model, a stochastic one is generated, capable of capturing and reproducing uncertain system–environment interactions at given levels of fidelity. The reported approach combines methodological elements from probabilistic model validation and randomized algorithms, to simultaneously quantify the fidelity of a model and tune the distribution of random parameters in the corresponding stochastic extension, in order to reproduce the variability observed experimentally in the physical process of interest. The approach can be applied to an array of physical processes, the models of which may come in different forms, including differential equations; we demonstrate this point by considering examples from the areas of miniature legged robots and aerial vehicles.

Probabilistic model validation for uncertain nonlinear systems

This paper presents a probabilistic model validation methodology for nonlinear systems in time-domain. The proposed formulation is simple, intuitive, and accounts both deterministic and stochastic nonlinear systems with parametric and nonparametric uncertainties. Instead of hard invalidation methods available in the literature, a relaxed notion of validation in probability is introduced. To guarantee provably correct inference, algorithm for constructing probabilistically robust validation certificate is given along with computational complexities. Several examples are worked out to illustrate its use.

Validation of Probabilistic Model for Design of Freeway Entrance Speed Change Lanes

Traffic operation on entrance speed change lanes (SCLs) is more complicated than on exit SCLs as vehicles interact constantly while merging from the SCL to the freeway. An adequate SCL length gives road users sufficient time to accelerate and to search for a gap so that they can comfortably merge onto the freeway. This study focused on analyzing entrance SCLs from a safety point of view in a probabilistic framework. Sixteen entrance-limited-type SCLs along Highway 417 were considered to investigate the link between the probabilistic measure, probability of noncompliance ( Pnc), and collision frequency. Regression analysis was conducted to quantify the effect of the geometry of the SCL, driver merging behavior along the SCL, road users’ exposure, and Pnc on collision occurrence. The model containing Pnc outperformed other models, and the inclusion of Pnc in the regression models resulted in significant improvement in the model fit. The study provided positive evidence on the validity of Pnc as a surrogate measure of safety.

Exploiting the information content of hydrological ''outliers'' for goodness-of-fit testing

Abstract. Validation of probabilistic models based on goodness-of-fit tests is an essential step for the frequency analysis of extreme events. The outcome of standard testing techniques, however, is mainly determined by the behavior of the hypothetical model, FX(x), in the central part of the distribution, while the behavior in the tails of the distribution, which is indeed very relevant in hydrological applications, is relatively unimportant for the results of the tests. The maximum-value test, originally proposed as a technique for outlier detection, is a suitable, but seldom applied, technique that addresses this problem. The test is specifically targeted to verify if the maximum (or minimum) values in the sample are consistent with the hypothesis that the distribution FX(x) is the real parent distribution. The application of this test is hindered by the fact that the critical values for the test should be numerically obtained when the parameters of FX(x) are estimated on the same sample used for verification, which is the standard situation in hydrological applications. We propose here a simple, analytically explicit, technique to suitably account for this effect, based on the application of censored L-moments estimators of the parameters. We demonstrate, with an application that uses artificially generated samples, the superiority of this modified maximum-value test with respect to the standard version of the test. We also show that the test has comparable or larger power with respect to other goodness-of-fit tests (e.g., chi-squared test, Anderson-Darling test, Fung and Paul test), in particular when dealing with small samples (sample size lower than 20–25) and when the parent distribution is similar to the distribution being tested.

Introduction to the Monte Carlo project and the approach to the validation of probabilistic models of dietary exposure to selected food chemicals

The Monte Carlo project was established to allow an international collaborative effort to define conceptual models for food chemical and nutrient exposure, to define and validate the software code to govern these models, to provide new or reconstructed databases for validation studies, and to use the new software code to complete validation modelling. Models were considered valid when they provided exposure estimates (ea) that could be shown not to underestimate the true exposure (eb), but at the same time are more realistic than the currently used conservative estimates (ec). Thus, validation required eb⩽ea<ec. In the case of pesticides, validation involved the collection of duplicate diets from 500 infants for pesticide analysis. In the case of intense sweeteners, a new consumption dataset was created among prescreened high consumers of intense sweeteners by recording, at brand level, all foods and beverages ingested over 12 days. In the case of nutrients and additives, existing databases were modified to minimize uncertainty over the model parameters. In most instances, it was possible to generate probabilistic models that fulfilled the validation criteria.