Conservative Confidence Research Articles

Standard Errors for Calibrated Parameters

Abstract Calibration, the practice of choosing the parameters of a structural model to match certain empirical moments, can be viewed as minimum distance estimation. Existing standard error formulas for such estimators require a consistent estimate of the correlation structure of the empirical moments, which is often unavailable in practice. Instead, the variances of the individual empirical moments are usually readily estimable. Using only these variances, we derive conservative standard errors and confidence intervals for the structural parameters that are valid even under the worst-case correlation structure. In the over-identified case, we show that the moment weighting scheme that minimizes the worst-case estimator variance amounts to a moment selection problem with a simple solution. Finally, we develop tests of over-identifying or parameter restrictions. We apply our methods empirically to a model of menu cost pricing for multi-product firms and to a heterogeneous agent New Keynesian model.

Reframing Confidence Instructions to Child Eyewitness Reduces Overconfidence but Does Not Improve Confidence–Accuracy Calibration

ABSTRACTChildren are well‐documented to exhibit poor confidence–accuracy calibration on lineup identification tasks. Children tend to report overconfidence in their (often inaccurate) lineup identification decisions. This research explored the extent to which school‐aged children's (N = 142; 6‐ to 8‐year‐old) confidence reports are implicitly driven by perceived social pressure to provide a specific confidence rating. Children were randomly assigned to two different confidence instruction conditions: the neutral (n = 69) or the reframed conditions (n = 73). The reframed instructions encouraged honesty and instructed children to ignore perceived pressure when reporting confidence. Results revealed that the reframed instructions resulted in more conservative confidence judgments; however, this shift did not translate into those confidence ratings better reflecting children's identification accuracy. Overall, these findings provide evidence that, while external or social factors play a contributing role, other aspects of development are likely contributing more to the poor confidence–accuracy calibration observed with child eyewitnesses.

Optimizing decision-making in uncertain environments through analysis of stochastic stationary Multi-Armed Bandit algorithms

Reinforcement learning traditionally plays a pivotal role in artificial intelligence and various practical applications, focusing on the interaction between an agent and its environment. Within this broad field, the multi-armed bandit (MAB) problem represents a specific subset, characterized by a sequential interaction between a learner and an environment where the agents actions do not alter the environment or reward distributions. MABs are prevalent in recommendation systems and advertising and are increasingly applied in sectors like agriculture and adaptive clinical trials. The stochastic stationary bandit problem, a fundamental category of MAB, is the primary focus of this article. Here, we delve into the implementation and analytical comparison of several key bandit algorithmsincluding Explore-then-Commit (ETC), Upper Confidence Bound (UCB), Thompson Sampling (TS), Epsilon-Greedy (-Greedy), SoftMax, and Conservative Lower Confidence Bound (CON-LCB)across various datasets. These datasets vary in the number of options (arms), reward distributions, and specific parameters, offering a broad testing ground. Additionally, this work provides an overview of the diverse applications of bandit problems across different fields, highlighting their versatility and broad impact.

Evaluating a novel 24-hour rest/activity rhythm marker of preclinical β-amyloid deposition.

To compare sleep and 24-hour rest/activity rhythms (RARs) between cognitively normal older adults who are β-amyloid-positive (Aβ+) or Aβ- and replicate a novel time-of-day-specific difference between these groups identified in a previous exploratory study. We studied 82 cognitively normal participants from the Baltimore Longitudinal Study of Aging (aged 75.7 ± 8.5 years, 55% female, 76% white) with wrist actigraphy data and Aβ+ versus Aβ- status measured by [11C] Pittsburgh compound B positron emission tomography. RARs were calculated using epoch-level activity count data from actigraphy. We used novel, data-driven function-on-scalar regression analyses and standard RAR metrics to cross-sectionally compare RARs between 25 Aβ+ and 57 Aβ- participants. Compared to Aβ- participants, Aβ+ participants had higher mean activity from 1:00 p.m. to 3:30 p.m. when using less conservative pointwise confidence intervals (CIs) and from 1:30 p.m. to 2:30 p.m. using more conservative, simultaneous CIs. Furthermore, Aβ+ participants had higher day-to-day variability in activity from 9:00 a.m. to 11:30 a.m. and lower variability from 1:30 p.m. to 4:00 p.m. and 7:30 p.m. to 10:30 p.m. according to pointwise CIs, and lower variability from 8:30 p.m. to 10:00 p.m. using simultaneous CIs. There were no Aβ-related differences in standard sleep or RAR metrics. Findings suggest Aβ+ older adults have higher, more stable day-to-day afternoon/evening activity than Aβ- older adults, potentially reflecting circadian dysfunction. Studies are needed to replicate our findings and determine whether these or other time-of-day-specific RAR features have utility as markers of preclinical Aβ deposition and if they predict clinical dementia and agitation in the afternoon/evening (i.e. "sundowning").

On positive association of absolute-valued and squared multivariate Gaussians beyond MTP2

We show that positively associated squared (and absolute-valued) multivariate normally distributed random vectors need not be multivariate totally positive of order 2 (MTP2) for p≥3. This result disproves Theorem 1 in Eisenbaum (2014, Ann. Probab.) and the conjecture that positive association of squared multivariate normals is equivalent to MTP2 and infinite divisibility of squared multivariate normals. Among others, we show that there exist absolute-valued multivariate normals which are conditionally increasing in sequence (CIS) (or weakly CIS (WCIS)) and hence positively associated but not MTP2. Moreover, we show that there exist absolute-valued multivariate normals which are positively associated but not CIS. As a by-product, we obtain necessary conditions for CIS and WCIS of absolute normals. We illustrate these conditions in some examples. With respect to implications and applications of our results, we show PA beyond MTP2 for some related multivariate distributions (chi-square, t, skew normal) and refer to possible conservative multiple test procedures and conservative simultaneous confidence bounds. Finally, we obtain the validity of the strong form of Gaussian product inequalities beyond MTP2.

Risk Assessment of Isoeugenol in Food Based on Benchmark Dose-Response Modeling.

Isoeugenol has recently been evaluated as possibly carcinogenic (Group 2B) by the WHO International Agency for Research on Cancer (IARC). In light of this evaluation, an updated risk assessment of this common food constituent was conducted using the benchmark dose (BMD) approach as recommended by the European Food Safety Authority (EFSA) for point of departure (POD) determination, as an alternative to the no observed adverse effect level (NOAEL). This approach was specifically chosen, as for the relevant neoplastic endpoints only lowest observed adverse effect level (LOAEL) values are available. The toxicological endpoint from the animal studies with the most conservative BMD lower confidence limit (BMDL) value was identified. Using the obtained BMDL value of 8 mg/kg body weight/day as POD, an acceptable daily intake (ADI) of 16 µg/kg body weight/day was obtained, which-despite being more conservative than previous approaches-is still clearly above the estimated daily exposure level to isoeugenol in the USA and in Europe. These results confirm a low risk of the estimated daily exposure levels of isoeugenol.

A compact multi-planet system transiting HIP 29442 (TOI-469) discovered by TESS and ESPRESSO

Context. One of the goals of the Echelle Spectrograph for Rocky Exoplanets and Stable Spectroscopic Observations (ESPRESSO) Guaranteed Time Observations (GTO) consortium is the precise characterisation of a selected sample of planetary systems discovered by TESS. One such target is the K0V star HIP 29442 (TOI-469), already known to host a validated sub-Neptune companion TOI-469.01, which we followed-up with ESPRESSO. Aims. We aim to verify the planetary nature of TOI-469.01 by obtaining precise mass, radius, and ephemeris, and constraining its bulk physical structure and composition. Methods. Following a Bayesian approach, we modelled radial velocity and photometric time series to measure the dynamical mass, radius, and ephemeris, and to characterise the internal structure and composition of TOI-469.01. Results. We confirmed the planetary nature of TOI-469.01 (now renamed HIP 29442 b), and thanks to the ESPRESSO radial velocities we discovered two additional close-in companions. Through an in-depth analysis of the TESS light curve, we could also detect their low signal-to-noise transit signals. We characterised the additional companions, and conclude that HIP 29442 is a compact multi-planet system. The three planets have orbital periods Porb,b = 13.63083 ± 0.00003, Porb,c = 3.53796 ± 0.00003, and Porb,d = 6.42975−0.00010+0.00009 days, and we measured their masses with high precision: mp,b = 9.6 ± 0.8 M⊕, mp,c = 4.5 ± 0.3 M⊕, and mp,d = 5.1 ± 0.4 M⊕. We measured radii and bulk densities of all the planets (the 3σ confidence intervals are shown in parentheses): Rp,b = 3.48−0.08(−0.28)+0.07(+0.19) R⊕ and ρp,b = 1.3 ± 0.2(0.3)g cm−3; Rp,c = 1.58−0.11(−0.34)+0.10(+0.30) R⊕ and ρp,c = 6.3−1.3(−2.7)+1.7(+6.0)g cm−3; Rp,d = 1.37 ± 0.11(−0.43)(+0.32) R⊕ and ρp,d = 11.0−2.4(−6.3)+3.4(+21.0)g cm−3. Due to noisy light curves, we used the more conservative 3σ confidence intervals for the radii as input to the interior structure modelling. We find that HIP 29442 b appears as a typical sub-Neptune, likely surrounded by a gas layer of pure H-He with amass of 0.27−0.17+0.24 M⊕ and a thickness of 1.4 ± 0.5 R⊕. For the innermost companions HIP 29442 c and HIP 29442 d, the model supports an Earth-like composition. Conclusions. The compact multi-planet system orbiting HIP 29442 offers the opportunity to study simultaneously planets straddling the gap in the observed radius distribution of close-in small-size exoplanets. High-precision photometric follow-up is required to obtain more accurate and precise radius measurements, especially for planets c and d. This, together with our determined high-precision masses, will provide the accurate and precise bulk structure of the planets, and enable an accurate investigation of the system’s evolution.

Demonstrating software reliability using possibly correlated tests: Insights from a conservative Bayesian approach

AbstractThis paper presents Bayesian techniques for conservative claims about software reliability, particularly when evidence suggests the software's executions are not statistically independent. We formalise informal notions of “doubting” that the executions are independent, and incorporate such doubts into reliability assessments. We develop techniques that reveal the extent to which independence assumptions can undermine conservatism in assessments, and identify conditions under which this impact is not significant. These techniques – novel extensions of conservative Bayesian inference (CBI) approaches – give conservative confidence bounds on the software's failure probability per execution. With illustrations in two application areas – nuclear power‐plant safety and autonomous vehicle (AV) safety – our analyses reveals: (1) the confidence an assessor should possess before subjecting a system to operational testing. Otherwise, such testing is futile – favourable operational testing evidence will eventually decrease one's confidence in the system being sufficiently reliable; (2) the independence assumption supports conservative claims sometimes; (3) in some scenarios, observing a system operate without failure gives less confidence in the system than if some failures had been observed; (4) building confidence in a system is very sensitive to failures – each additional failure means significantly more operational testing is required, in order to support a reliability claim.

Adaptive Multiple Comparison Sequential Design (AMCSD) for clinical trials

ABSTRACT We propose an adaptive sequential testing procedure for clinical trials that test the efficacy of multiple treatment options, such as dose/regimen, different drugs, sub-populations, endpoints, or a mixture of them in one trial. At any interim analyses, sample size re-estimation can be conducted, and any option can be dropped for lack of efficacy or unsatisfactory safety profile. Inference after the trial, including p-value, conservative point estimate and confidence intervals, are provided.

Adsorption of praziquantel enantiomers on chiral cellulose tris 3-chloro, 4-methylphenylcarbamate by frontal analysis: Fisherian and Bayesian parameter estimation and inference

Praziquantel (PZQ) is an anthelmintic chiral pharmaceutical utilized in schistosomiasis treatment, commonly sold as a racemate, whose primary active molecule is the enantiomer L-(-)-PZQ. The development of new pharmaceutical formulations contenting L-(-)-PZQ has mobilized worldwide efforts from the academy and private companies. Several processes have been proposed to produce pure L-(-)-PZQ, including racemate resolution by preparative chromatography. The design of complex chromatographic processes such as SMB requires accurate information about the adsorption isotherm models and other system parameters and well-quantified uncertainties. We obtained the adsorption isotherms of both PZQ enantiomers using the Frontal Analysis (FA) technique. The associated uncertainties and model confidence bands were calculated from Fisherian and Bayesian approaches. Parameter uncertainties from both methods presented reasonable agreement. Bayesian inference allowed calculating conservative confidence intervals for the parameters, the isotherm curves and the experimental profiles related to FA. Predicted confidence intervals varied from 5.6% to 14% for parameters, 3.9% to 7.1% for the isotherms and 2.02% to 2.22% for the concentration on FA profiles. The estimated nuisance factor agreed with the experimental relative standard deviation and could be applied to predict experimental variances when the same is absent.

Bootstrap Confidence Intervals for Common Signal-to-noise Ratio of Two-parameter Exponential Distributions

Signal-to-noise ratio (SNR) is a reciprocal of coefffficient of variation. The SNR is a measure of mean relative to the variability. Confidence procedures for common SNR of two-parameter exponential distributions were developed using generalized confidence interval (GCI) approach, large sample (LS) approach, adjusted method of variance estimates recovery (Adjusted MOVER) approach, and bootstrap approaches based on standard bootstrap (SB) and parametric bootstrap (PB). The performances of all approaches are measured by coverage probability and average length. Simulation studies show that all approaches have the coverage probabilities below the nominal confidence level of 0.95 when the common SNR is negative value. However, the coverage probabilities of all approaches are greater than the nominal confidence level of 0.95 when the common SNR is positive value. Moreover, the LS and AM approaches are the conservative confidence intervals. In addition, the GCI and PB approaches provide the confidence intervals with coverage probabilities close to the nominal confidence level of 0.95 when the sample sizes are large and the common SNR is positive value. The GCI and PB approaches are recommended to estimate the confidence intervals for the common SNR of two-parameter exponential distributions. Finally, all proposed approaches are employed in the data of the survival days of lung cancer patients for a demonstration.

Efficient computation of tight approximations to Chernoff bounds

Chernoff bounds are a powerful application of the Markov inequality to produce strong bounds on the tails of probability distributions. They are often used to bound the tail probabilities of sums of Poisson trials, or in regression to produce conservative confidence intervals for the parameters of such trials. The bounds provide expressions for the tail probabilities that can be inverted for a given probability/confidence to provide tail intervals. The inversions involve the solution of transcendental equations and it is often convenient to substitute approximations that can be exactly solved e.g. by the quadratic equation. In this paper we introduce approximations for the Chernoff bounds whose inversion can be exactly solved with a quadratic equation, but which are closer approximations than those adopted previously.

Corporate failure prediction using threshold‐based models

AbstractCorporate failure prediction literature indicates that models' performance depends on more than the complexity of the prediction method. Recent advances cite the relevance of sampling approaches to model performance. Therefore, this study proposes a novel approach that implements threshold models for corporate failure prediction efforts. Threshold models estimate asymptotically conservative confidence regions, in which the samples are split by size. Then, single‐based classifiers and a combination of multiple classifiers are employed in each region to estimate the prediction accuracy. This article offers a comparison of the proposed threshold‐based corporate failure model with two benchmark models, previously used in studies in the same context. The empirical results show that the proposed threshold‐based model clearly outperforms conventional models, in particular with the combination of multiple classifiers. The superiority of the threshold‐based model stems from its ability to discern failed firms, which represent the most important class in financial terms. This study thus provides initial evidence of the utility of threshold models in corporate failure prediction efforts.

Stacked Grenander and rearrangement estimators of a discrete distribution

In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with n-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodness-of-fit and shape constraints.

A penalization approach to random-effects meta-analysis.

Systematic reviews and meta-analyses are principal tools to synthesize evidence from multiple independent sources in many research fields. The assessment of heterogeneity among collected studies is a critical step when performing a meta-analysis, given its influence on model selection and conclusions about treatment effects. A common-effect (CE) model is conventionally used when the studies are deemed homogeneous, while a random-effects (RE) model is used for heterogeneous studies. However, both models have limitations. For example, the CE model produces excessively conservative confidence intervals with low coverage probabilities when the collected studies have heterogeneous treatment effects. The RE model, on the other hand, assigns higher weights to small studies compared to the CE model. In the presence of small-study effects or publication bias, the over-weighted small studies from a RE model can lead to substantially biased overall treatment effect estimates. In addition, outlying studies may exaggerate between-study heterogeneity. This article introduces penalization methods as a compromise between the CE and RE models. The proposed methods are motivated by the penalized likelihood approach, which is widely used in the current literature to control model complexity and reduce variances of parameter estimates. We compare the existing and proposed methods with simulated data and several case studies to illustrate the benefits of the penalization methods.

Rerandomization in Stratified Randomized Experiments

Stratification and rerandomization are two well-known methods used in randomized experiments for balancing the baseline covariates. Renowned scholars in experimental design have recommended combining these two methods; however, limited studies have addressed the statistical properties of this combination. This article proposes two rerandomization methods to be used in stratified randomized experiments, based on the overall and stratum-specific Mahalanobis distances. The first method is applicable for nearly arbitrary numbers of strata, strata sizes, and stratum-specific proportions of the treated units. The second method, which is generally more efficient than the first method, is suitable for situations in which the number of strata is fixed with their sizes tending to infinity. Under the randomization inference framework, we obtain the asymptotic distributions of estimators used in these methods and the formulas of variance reduction when compared to stratified randomization. Our analysis does not require any modeling assumption regarding the potential outcomes. Moreover, we provide asymptotically conservative variance estimators and confidence intervals for the average treatment effect. The advantages of the proposed methods are exhibited through an extensive simulation study and a real-data example.

A Distributionally Robust Chance-Constrained Unit Commitment with N-1 Security and Renewable Generation

With the increasing penetration of renewable energy generation, one of the major challenges is the problem of how to express the stochastic process of wind power and photovoltaic output as the exact probability density and distribution, in order to improve the security and accuracy of unit commitment results, a distributed robust security-constrained optimization model based on moment uncertainty is proposed, in which the uncertainty of wind and photovoltaic power is captured by two uncertain sets of first- and second-order moments, respectively. The two sets contain the probability distribution of the forecast error of the wind and photovoltaic power, and in the model, the energy storage is considered. In order to solve the model effectively, firstly, based on the traditional chance-constrained second-order cone transformation, according to the first- and second-order moments polyhedron expression of the distribution set, a cutting plane method is proposed to solve the distributed robust chance constraints. Secondly, the modified IEEE-RTS 24 bus system is selected to establish a simulation example, an improved generalized Benders decomposition algorithm is developed to solve the model to optimality. The results show that the unit commitment results with different emphasis on economy and security can be obtained by setting different conservative coefficients and confidence levels and, then, provide a reasonable decision-making basis for dispatching operation.

Pseudoreplication in genomic-scale data sets.

In genomic-scale data sets, loci are closely packed within chromosomes and hence provide correlated information. Averaging across loci as if they were independent creates pseudoreplication, which reduces the effective degrees of freedom (df') compared to the nominal degrees of freedom, df. This issue has been known for some time, but consequences have not been systematically quantified across the entire genome. Here, we measured pseudoreplication (quantified by the ratio df'/df) for a common metric of genetic differentiation (FST ) and a common measure of linkage disequilibrium between pairs of loci (r2 ). Based on data simulated using models (SLiM and msprime) that allow efficient forward-in-time and coalescent simulations while precisely controlling population pedigrees, we estimated df' and df'/df by measuring the rate of decline in the variance of mean FST and mean r2 as more loci were used. For both indices, df' increases with Ne and genome size, as expected. However, even for large Ne and large genomes, df' for mean r2 plateaus after a few thousand loci, and a variance components analysis indicates that the limiting factor is uncertainty associated with sampling individuals rather than genes. Pseudoreplication is less extreme for FST , but df'/df ≤0.01 can occur in data sets using tens of thousands of loci. Commonly-used block-jackknife methods consistently overestimated var (FST ), producing very conservative confidence intervals. Predicting df' based on our modelling results as a function of Ne , L, S, and genome size provides a robust way to quantify precision associated with genomic-scale data sets.

Conservative Confidence Interval Prediction in Fused Deposition Modeling Process With Linear Optimization Approach

Abstract Regression models are widely used as data-driven methods for predicting a continuous target variable. From a set of input variables, regression models predict a deterministic point value for the target variable. But the deterministic point value prediction is not always sufficient because a target variable value often varies due to diverse sources of uncertainty. For instance, in the fused deposition modeling process, the inconsistent results of replications are associated with natural randomness, environmental condition, and noisy process parameters. The point value estimation is not sufficient to represent the variability in this kind of scenario. Instead of point estimation, the interval prediction of a target variable is more useful in this application. In this paper, linear optimization-based techniques are proposed to predict conservative confidence intervals for linear and polynomial regression models. Two linear optimization models, one for ordinary least squares (OLS) regression and the other for weighted least squares (WLS) regression, are proposed. The proposed methods are implemented on several datasets, including an experimental fused deposition modeling dataset to demonstrate the effectiveness of the proposed methods. The results show that the proposed method is useful for the fused deposition modeling process where the level of uncertainty or the lack of knowledge of uncertainty sources is high. The proposed method can also be leveraged to the Bayesian neural network (BNN), where the optimization techniques for interval prediction will be nonlinear optimization instead of linear optimization.

Nonparametric Modelling of Quarterly Unemployment Rates

A seasonal additive nonlinear vector autoregression (SANVAR) model is proposed for multivariate seasonal time series to explore the possible interaction among the various univariate series. Significant lagged variables are selected and additive autoregression functions estimated based on the selected variables using spline smoothing method. Conservative confidence bands are constructed for the additive autoregression function. The model is fitted to two sets of bivariate quarterly unemployment rate data with comparisons made to the linear periodic vector autoregression model. It is found that when the data does not significantly deviate from linearity, the periodic model is preferred. In cases of strong nonlinearity, however, the additive model is more parsimonious and has much higher out-of-sample prediction power. In addition, interactions among various univariate series are automatically detected.