Measures Of Statistical Uncertainty Research Articles

Bayesian Inventory Control: Accelerated Demand Learning via Exploration Boosts

In the Bayesian newsvendor problem, it is known that the optimal decision is always greater than or equal to the myopic decision. As a result, the optimal decision can be expressed as the sum of the myopic decision plus a nonnegative “exploration boost.” In “Bayesian Inventory Control: Accelerated Demand Learning via Exploration Boosts,” Chuang and Kim characterize the form of the exploration boost in terms of basic statistical measures of uncertainty. This characterization expresses in clear terms the way in which the statistical learning and inventory control are jointly optimized; when there is a high degree of parameter uncertainty, inventory levels are boosted to induce a higher chance of observing more sales data to more quickly resolve statistical uncertainty, and as parameter uncertainty resolves, the exploration boost is reduced.

Confidence bands for a log-concave density

We present a new approach for inference about a univariate log-concave distribution: Instead of using the method of maximum likelihood, we propose to incorporate the log-concavity constraint in an appropriate nonparametric confidence set for the cdf F. This approach has the advantage that it automatically provides a measure of statistical uncertainty and it thus overcomes a marked limitation of the maximum likelihood estimate. In particular, we show how to construct confidence bands for the density that have a finite sample guaranteed confidence level. The nonparametric confidence set for F which we introduce here has attractive computational and statistical properties: It allows to bring modern tools from optimization to bear on this problem via difference of convex programming, and it results in optimal statistical inference. We show that the width of the resulting confidence bands converges at nearly the parametric rate when the log density is k-affine.

Ensembles Are Required to Handle Aleatoric and Parametric Uncertainty in Molecular Dynamics Simulation.

Classical molecular dynamics is a computer simulation technique that is in widespread use across many areas of science, from physics and chemistry to materials, biology, and medicine. The method continues to attract criticism due its oft-reported lack of reproducibility which is in part due to a failure to submit it to reliable uncertainty quantification (UQ). Here we show that the uncertainty arises from a combination of (i) the input parameters and (ii) the intrinsic stochasticity of the method controlled by the random seeds. To illustrate the situation, we make a systematic UQ analysis of a widely used molecular dynamics code (NAMD), applied to estimate binding free energy of a ligand-bound to a protein. In particular, we replace the usually fixed input parameters with random variables, systematically distributed about their mean values, and study the resulting distribution of the simulation output. We also perform a sensitivity analysis, which reveals that, out of a total of 175 parameters, just six dominate the variance in the code output. Furthermore, we show that binding energy calculations dampen the input uncertainty, in the sense that the variation around the mean output free energy is less than the variation around the mean of the assumed input distributions, if the output is ensemble-averaged over the random seeds. Without such ensemble averaging, the predicted free energy is five times more uncertain. The distribution of the predicted properties is thus strongly dependent upon the random seed. Owing to this substantial uncertainty, robust statistical measures of uncertainty in molecular dynamics simulation require the use of ensembles in all contexts.

Measures of global sensitivity in linear programming: applications in banking sector

The paper examines the sensitivity for the solution of linear programming problems using Bayesian techniques, when samples for the coefficients of the objective function are uncertain. When data is available, we estimate the solution of the linear program and provide statistical measures of uncertainty through the posterior distributions of the solution in the light of the data. When data is not available, these techniques examine the sensitivity of the solution to random variation in the coefficients of the linear problem. The new techniques are based on two posteriors emerging from the inequalities of Karush–Kuhn–Tucker conditions. The first posterior is asymptotic and does not require data. The second posterior is finite-sample-based and is used whenever data is available or if random samples can be drawn from the joint distribution of coefficients. A by-product of our framework is a robust solution. We illustrate the new techniques in two empirical applications to the case of uncertain Data Envelopment Analysis efficiency, involving two large samples, of US commercial banks and a sample of European commercial banks regulated by the Single Supervisory Mechanism. We analyse whether some pre-determined criteria, associated with size and new supervisory framework, can adequately affect the solution of linear program. The results provide evidence of substantial improvements in statistical structure with respect to sensitivities and robustification. Our methodology can serve as a consistency check of the statistical inference for the solution of linear programming problems in efficiency under uncertainty in data.

Quantifying Uncertainty in Mechanistic Models of Infectious Disease.

This primer describes the statistical uncertainty in mechanistic models and provides R code to quantify it. We begin with an overview of mechanistic models for infectious disease, and then describe the sources of statistical uncertainty in the context of a case study on SARS-CoV-2. We describe the statistical uncertainty as belonging to three categories: data uncertainty, stochastic uncertainty, and structural uncertainty. We demonstrate how to account for each of these via statistical uncertainty measures and sensitivity analyses broadly, as well as in a specific case study on estimating the basic reproductive number, n}{}{R}_0, for SARS-CoV-2.

Assessing Uncertainty in the Rooting of the SARS-CoV-2 Phylogeny.

The rooting of the SARS-CoV-2 phylogeny is important for understanding the origin and early spread of the virus. Previously published phylogenies have used different rootings that do not always provide consistent results. We investigate several different strategies for rooting the SARS-CoV-2 tree and provide measures of statistical uncertainty for all methods. We show that methods based on the molecular clock tend to place the root in the B clade, whereas methods based on outgroup rooting tend to place the root in the A clade. The results from the two approaches are statistically incompatible, possibly as a consequence of deviations from a molecular clock or excess back-mutations. We also show that none of the methods provide strong statistical support for the placement of the root in any particular edge of the tree. These results suggest that phylogenetic evidence alone is unlikely to identify the origin of the SARS-CoV-2 virus and we caution against strong inferences regarding the early spread of the virus based solely on such evidence.

Generalized Additive Mixed Modeling of Longitudinal Tumor Growth Reduces Bias and Improves Decision Making in Translational Oncology

Scientists working in translational oncology regularly conduct multigroup studies of mice with serially measured tumors. Longitudinal data collected can feature mid-study dropouts and complex nonlinear temporal response patterns. Parametric statistical models such as ones assuming exponential growth are useful for summarizing tumor volume over ranges for which the growth model holds, with the advantage that the model's parameter estimates can be used to summarize between-group differences in tumor volume growth with statistical measures of uncertainty. However, these same assumed growth models are too rigid to recapitulate patterns observed in many experiments, which in turn diminishes the effectiveness of their parameter estimates as summary statistics. To address this problem, we generalized such models by adopting a nonparametric approach in which group-level response trends for logarithmically scaled tumor volume are estimated as regression splines in a generalized additive mixed model. We also describe a novel summary statistic for group level splines over user-defined, experimentally relevant time ranges. This statistic reduces to the log-linear growth rate for data well described by exponential growth and also has a sampling distribution across groups that is well approximated by a multivariate Gaussian, thus facilitating downstream analysis. Real-data examples show that this nonparametric approach not only enhances fidelity in describing nonlinear growth scenarios but also improves statistical power to detect interregimen differences when compared with the simple exponential model so that it generalizes the linear mixed effects paradigm for analysis of log-linear growth to nonlinear scenarios in a useful way. SIGNIFICANCE: This work generalizes the statistical linear mixed modeling paradigm for summarizing longitudinally measured preclinical tumor volume studies to encompass studies with nonlinear and nonmonotonic group response patterns in a statistically rigorous manner.

Predicting uncertainty of machine learning models for modelling nitrate pollution of groundwater using quantile regression and UNEEC methods

Although estimating the uncertainty of models used for modelling nitrate contamination of groundwater is essential in groundwater management, it has been generally ignored. This issue motivates this research to explore the predictive uncertainty of machine-learning (ML) models in this field of study using two different residuals uncertainty methods: quantile regression (QR) and uncertainty estimation based on local errors and clustering (UNEEC). Prediction-interval coverage probability (PICP), the most important of the statistical measures of uncertainty, was used to evaluate uncertainty. Additionally, three state-of-the-art ML models including support vector machine (SVM), random forest (RF), and k-nearest neighbor (kNN) were selected to spatially model groundwater nitrate concentrations. The models were calibrated with nitrate concentrations from 80 wells (70% of the data) and then validated with nitrate concentrations from 34 wells (30% of the data). Both uncertainty and predictive performance criteria should be considered when comparing and selecting the best model. Results highlight that the kNN model is the best model because not only did it have the lowest uncertainty based on the PICP statistic in both the QR (0.94) and the UNEEC (in all clusters, 0.85–0.91) methods, but it also had predictive performance statistics (RMSE = 10.63, R2 = 0.71) that were relatively similar to RF (RMSE = 10.41, R2 = 0.72) and higher than SVM (RMSE = 13.28, R2 = 0.58). Determining the uncertainty of ML models used for spatially modelling groundwater-nitrate pollution enables managers to achieve better risk-based decision making and consequently increases the reliability and credibility of groundwater-nitrate predictions.

Quantifying statistical uncertainty in metrics of sleep disordered breathing

BackgroundThe apnea-hypopnea index (AHI) (or one of its derivatives) is the primary clinical metric for characterizing sleep disordered breathing—the value of which with respect to a threshold determines severity of diagnosis and eligibility for treatment reimbursement. The index value, however, is taken as a perfect point estimate, with no measure of statistical uncertainty. Thus, current practice does not robustly account for variability in diagnosis/eligibility due to chance. In this paper, we quantify the statistical uncertainty associated with respiratory event indices for sleep disordered breathing and the effect of uncertainty on treatment eligibility. MethodsWe develop an empirical estimate of uncertainty using a non-parametric bootstrap on the interevent times, as well as a theoretical Poisson estimate reflecting the current formulation of the AHI. We then apply these methods to estimate AHI uncertainty for 2049 subjects (954/1095 M/F, age: mean 69 ± 9.1) from the Multi-Ethnic Study of Atherosclerosis (MESA). Results and ConclusionsThe mean 95% empirical confidence interval width was 11.500 ± 6.208 events per hour and the mean 95% theoretical Poisson confidence interval width was 5.998 ± 2.897 events per hour, suggesting that uncertainty is likely a major confounding factor within the current diagnostic framework. Of the 278 subjects in the symptomatic population (ESS>10), 27% (76/278) had uncertain diagnoses given the 95% empirical confidence interval. Of the 2049 subjects in the full population, 43% (880/2049) had uncertain diagnoses given the 95% empirical confidence interval. The inclusion of subjects with uncertain diagnoses increases the number of eligible patients by 21.3% for the symptomatic population and by 84.8% for the full population. The exclusion of subjects with uncertain diagnoses given the 95% empirical confidence interval decreases the number of eligible patients by 12.4% for the symptomatic population and by 34.8% for full population. Additional analyses suggest that it is practically infeasible to gain diagnostic statistical significance through additional testing for a broad range of borderline cases. Overall, these results suggest that AHI uncertainty is a vital additional piece of information that would greatly benefit clinical practice, and that the inclusion of uncertainty in epidemiological analysis might help improve the ability for researchers to robustly link AHI with co-morbidities and long-term outcomes.

Complexity in hydroecological modelling: A comparison of stepwise selection and information theory

AbstractUnderstanding of the hydroecological relationship is vital to maintaining the health of the river and thus its ecosystem. Stepwise selection is widely used to develop numerical models which represent these processes. Increasingly, however, there are questions over the suitability of the approach, and coupled with the increasing complexity of hydroecological modelling, there is a real need to consider alternative approaches. In this study, stepwise selection and information theory are employed to develop models which represent two realizations of the system which recognizes increasing complexity. The two approaches are assessed in terms of model structure, modelling error, and model (statistical) uncertainty. The results appear initially inconclusive, with the information theory approach leading to a reduction in modelling error but greater uncertainty. A Monte Carlo approach, used to explore this uncertainty, revealed modelling errors to be only slightly more distributed for the information theory approach. Consideration of the philosophical underpinnings of the two approaches provides greater clarity. Statistical uncertainty, as measured by information theory, will always be greater due to its consideration of two sources, parameter and model selection. Consequently, by encompassing greater information, the measure of statistical uncertainty is more realistic, making an information theory approach more reflective of the complexity in real‐world applications.

Quantifying and reducing statistical uncertainty in sample-based health program costing studies in low- and middle-income countries.

Objectives:In many low- and middle-income countries, the costs of delivering public health programs such as for HIV/AIDS, nutrition, and immunization are not routinely tracked. A number of recent studies have sought to estimate program costs on the basis of detailed information collected on a subsample of facilities. While unbiased estimates can be obtained via accurate measurement and appropriate analyses, they are subject to statistical uncertainty. Quantification of this uncertainty, for example, via standard errors and/or 95% confidence intervals, provides important contextual information for decision-makers and for the design of future costing studies. While other forms of uncertainty, such as that due to model misspecification, are considered and can be investigated through sensitivity analyses, statistical uncertainty is often not reported in studies estimating the total program costs. This may be due to a lack of awareness/understanding of (1) the technical details regarding uncertainty estimation and (2) the availability of software with which to calculate uncertainty for estimators resulting from complex surveys. We provide an overview of statistical uncertainty in the context of complex costing surveys, emphasizing the various potential specific sources that contribute to overall uncertainty.Methods:We describe how analysts can compute measures of uncertainty, either via appropriately derived formulae or through resampling techniques such as the bootstrap. We also provide an overview of calibration as a means of using additional auxiliary information that is readily available for the entire program, such as the total number of doses administered, to decrease uncertainty and thereby improve decision-making and the planning of future studies.Results:A recent study of the national program for routine immunization in Honduras shows that uncertainty can be reduced by using information available prior to the study. This method can not only be used when estimating the total cost of delivering established health programs but also to decrease uncertainty when the interest lies in assessing the incremental effect of an intervention.Conclusion:Measures of statistical uncertainty associated with survey-based estimates of program costs, such as standard errors and 95% confidence intervals, provide important contextual information for health policy decision-making and key inputs for the design of future costing studies. Such measures are often not reported, possibly because of technical challenges associated with their calculation and a lack of awareness of appropriate software. Modern statistical analysis methods for survey data, such as calibration, provide a means to exploit additional information that is readily available but was not used in the design of the study to significantly improve the estimation of total cost through the reduction of statistical uncertainty.

Stress testing and model validation: application of the Bayesian approach to a credit risk portfolio

Following the recent global financial crisis, regulators have recognized the importance of stress testing, in part due to the impact of model risk, and have implemented supervisory requirements in both the revised Basel framework and the Comprehensive Capital Analysis and Review (CCAR) program. We contribute to the literature by developing a Bayesian-based credit risk stress-testing methodology, which can be implemented by small-to-medium-sized banks, as well as presenting empirical results using data from the recent CCAR implementations. Through the application of a Bayesian model, we can formally incorporate exogenous scenarios and also quantify the uncertainty in model output that results from stochastic model inputs. We contribute to the model validation literature by comparing the proportional model risk buffer measure of the severely adverse cumulative nine-quarter loss estimate - a common way to estimate, being a measure of statistical uncertainty generated by a model - obtained from our empirical implementation of the Bayesian to the frequentist model. We find it to be 40% higher in the former than in the latter. As for the model validation exercise, the Bayesian model outperforms the frequentist model statistically significantly, according to the cumulative percentage error metric, by 2% (1.5%) over the entire sample (downturn period).

Video saliency incorporating spatiotemporal cues and uncertainty weighting.

We propose a novel algorithm to detect visual saliency from video signals by combining both spatial and temporal information and statistical uncertainty measures. The main novelty of the proposed method is twofold. First, separate spatial and temporal saliency maps are generated, where the computation of temporal saliency incorporates a recent psychological study of human visual speed perception. Second, the spatial and temporal saliency maps are merged into one using a spatiotemporally adaptive entropy-based uncertainty weighting approach. The spatial uncertainty weighing incorporates the characteristics of proximity and continuity of spatial saliency, while the temporal uncertainty weighting takes into account the variations of background motion and local contrast. Experimental results show that the proposed spatiotemporal uncertainty weighting algorithm significantly outperforms state-of-the-art video saliency detection models.

The fossilized birth-death process for coherent calibration of divergence-time estimates.

Time-calibrated species phylogenies are critical for addressing a wide range of questions in evolutionary biology, such as those that elucidate historical biogeography or uncover patterns of coevolution and diversification. Because molecular sequence data are not informative on absolute time, external data--most commonly, fossil age estimates--are required to calibrate estimates of species divergence dates. For Bayesian divergence time methods, the common practice for calibration using fossil information involves placing arbitrarily chosen parametric distributions on internal nodes, often disregarding most of the information in the fossil record. We introduce the "fossilized birth-death" (FBD) process--a model for calibrating divergence time estimates in a Bayesian framework, explicitly acknowledging that extant species and fossils are part of the same macroevolutionary process. Under this model, absolute node age estimates are calibrated by a single diversification model and arbitrary calibration densities are not necessary. Moreover, the FBD model allows for inclusion of all available fossils. We performed analyses of simulated data and show that node age estimation under the FBD model results in robust and accurate estimates of species divergence times with realistic measures of statistical uncertainty, overcoming major limitations of standard divergence time estimation methods. We used this model to estimate the speciation times for a dataset composed of all living bears, indicating that the genus Ursus diversified in the Late Miocene to Middle Pliocene.

Measures of uncertainty in market network analysis

A general approach to measure statistical uncertainty of different filtration techniques for market network analysis is proposed. Two measures of statistical uncertainty are introduced and discussed. One is based on conditional risk for multiple decision statistical procedures and another one is based on average fraction of errors. It is shown that for some important cases the second measure is a particular case of the first one. The proposed approach is illustrated by numerical evaluation of statistical uncertainty for popular network structures (minimum spanning tree, planar maximally filtered graph, market graph, maximum cliques and maximum independent sets) in the framework of Gaussian network model of stock market.

Foreword: Conference Bias

Foreword: Conference Bias

Causal statistical inference in high dimensions

We present a short selective review of causal inference from observational data, with a particular emphasis on the high-dimensional scenario where the number of measured variables may be much larger than sample size. Despite major identifiability problems, making causal inference from observational data very ill-posed, we outline a methodology providing useful bounds for causal effects. Furthermore, we discuss open problems in optimization, non-linear estimation and for assigning statistical measures of uncertainty, and we illustrate the benefits and limitations of high-dimensional causal inference for biological applications.

Bovine tuberculosis: prevalence and diagnostic efficacy of routine meat inspection procedure in Woldiya municipality abattoir north Wollo zone, Ethiopia

Bovine tuberculosis (BTB) is a widespread and endemic disease of cattle in Ethiopia posing a significant threat to public health. Regular surveillance by skin test, bacteriology, and molecular methods is not feasible due to lack of resources. Thus, routine abattoir (RA) inspection will continue to play a key role for national surveillance. A cross-sectional study was conducted at Woldiya municipal abattoir from April 1, 2009 to April 5, 2010 to estimate the prevalence of BTB in slaughtered cattle on the basis of detailed abattoir inspection and to compare efficacy of RA inspection with respect to detailed abattoir inspection and isolation and identification of Mycobacterium. Diagnostic accuracies (with corresponding measures of statistical uncertainty) were determined by computing test property statistics (sensitivity and specificity). Agreement between RA and detailed abattoir inspections was measured using kappa statistics. Out of 1,029 slaughtered heads of cattle examined during the study period, 63 (6.12%) and 15 (1.45%) were diagnosed with gross tuberculous lesions by detailed abattoir meat inspections and RA meat inspections, respectively, making a prevalence of 6.12% (95% CI: 5.2-7.1) on the basis of detailed abattoir inspection. About 59.45% of tuberculous lesions were observed in the lungs and associated lymph nodes, whereas 35.13% lesions were from the lymph nodes of the head. From 63 cattle suspected with tuberculosis (TB) based on detailed abattoir meat inspection, nine (19.05%) were identified as Mycobacterium bovis, while three (4.8%) as Mycobacterium tuberculosis. The sensitivity of RA meat inspection was 23.8% in comparison to the detailed abattoir meat inspection and 25% in comparison to culture, respectively. Poor agreement (k = 0.37) was seen between RA meat examination and detailed abattoir meat examination methods. Similarly, poor agreement (k = 0.013) was seen between RA meat examination and culture results. In conclusion, relatively higher prevalence (6.12%) was recorded in Woldiya municipal abattoir on the basis of detailed Abattoir inspection and RA meat inspection protocols currently utilized in Ethiopia which are insufficient to detect the majority (76.19%) of TB lesions at the gross level, which indicates the magnitude of meat borne zoonotic TB as an ongoing risk to public health. Detailed abattoir inspection protocols were demonstrated to improve the detection level by approximately fourfold. In conclusion, routine meat inspections have limitations in detecting BTB-suggestive lesions which indicate the magnitude of meat-borne zoonotic TB as an ongoing risk to public health.

Finding Disease Variants in Mendelian Disorders By Using Sequence Data: Methods and Applications

Many sequencing studies are now underway to identify the genetic causes for both Mendelian and complex traits. Via exome-sequencing, genes harboring variants implicated in several Mendelian traits have already been identified. The underlying methodology in these studies is a multistep algorithm based on filtering variants identified in a small number of affected individuals and depends onwhether they are novel (not yet seen in public resources such as dbSNP), shared among affected individuals, and other external functional information on the variants. Although intuitive, these filter-based methods are nonoptimal and do not provide any measure of statistical uncertainty. We describe here a formal statistical approach that has several distinct advantages: (1) it provides fast computation of approximate p values for individual genes, (2) it adjusts for the background variation in each gene, (3) it allows for incorporation of functional or linkage-based information, and (4) it accommodates designs based on both affected relative pairs and unrelated affected individuals. We show via simulations that the proposed approach can be used in conjunction with the existing filter-based methods to achieve a substantially better ranking of a gene relevant for disease when compared to currently used filter-based approaches, this is especially so in the presence of disease locus heterogeneity. We revisit recent studies on three Mendeliandiseases and show that the proposed approach results in the implicated gene being ranked first in all studies, and approximate p values of 10(-6) for the Miller Syndrome gene, 1.0× 10(-4) for the Freeman-Sheldon Syndrome gene, and 3.5× 10(-5) for the Kabuki Syndrome gene.

Reconstructing Population Histories from Single Nucleotide Polymorphism Data

Population genetics encompasses a strong theoretical and applied research tradition on the multiple demographic processes that shape genetic variation present within a species. When several distinct populations exist in the current generation, it is often natural to consider the pattern of their divergence from a single ancestral population in terms of a binary tree structure. Inference about such population histories based on molecular data has been an intensive research topic in the recent years. The most common approach uses coalescent theory to model genealogies of individuals sampled from the current populations. Such methods are able to compare several different evolutionary scenarios and to estimate demographic parameters. However, their major limitation is the enormous computational complexity associated with the indirect modeling of the demographies, which limits the application to small data sets. Here, we propose a novel Bayesian method for inferring population histories from unlinked single nucleotide polymorphisms, which is applicable also to data sets harboring large numbers of individuals from distinct populations. We use an approximation to the neutral Wright-Fisher diffusion to model random fluctuations in allele frequencies. The population histories are modeled as binary rooted trees that represent the historical order of divergence of the different populations. A combination of analytical, numerical, and Monte Carlo integration techniques are utilized for the inferences. A particularly important feature of our approach is that it provides intuitive measures of statistical uncertainty related with the estimates computed, which may be entirely lacking for the alternative methods in this context. The potential of our approach is illustrated by analyses of both simulated and real data sets.