Frequentist Coverage Probabilities Research Articles

Improved confidence estimation for the binomial proportion with applications to clinical studies

ABSTRACT In this paper, we establish a frequentist framework that incorporates all confidence sets with guaranteed frequentist coverage probability for the binomial proportion, where different confidence sets are completely characterized by their tail functions. Based on measures of precision in the form of interval length, probability of false coverage, and a new evaluation criterion utilizing prior information, we construct the optimal confidence set for the binomial proportion. The newly proposed methodology is applied to clinical studies. It is shown that confidence intervals obtained via the tail functions are often better than prevailing confidence intervals in view of precision.

Real-time estimation and forecasting of COVID-19 cases and hospitalizations in Wisconsin HERC regions for public health decision making processes

BackgroundThe spread of the COVID-19 (SARS-CoV-2) and the surging number of cases across the United States have resulted in full hospitals and exhausted health care workers. Limited availability and questionable reliability of the data make outbreak prediction and resource planning difficult. Any estimates or forecasts are subject to high uncertainty and low accuracy to measure such components. The aim of this study is to apply, automate, and assess a Bayesian time series model for the real-time estimation and forecasting of COVID-19 cases and number of hospitalizations in Wisconsin healthcare emergency readiness coalition (HERC) regions.MethodsThis study makes use of the publicly available Wisconsin COVID-19 historical data by county. Cases and effective time-varying reproduction number R_t by the HERC region over time are estimated using Bayesian latent variable models. Hospitalizations are estimated by the HERC region over time using a Bayesian regression model. Cases, effective Rt, and hospitalizations are forecasted over a 1-day, 3-day, and 7-day time horizon using the last 28 days of data, and the 20%, 50%, and 90% Bayesian credible intervals of the forecasts are calculated. The frequentist coverage probability is compared to the Bayesian credible level to evaluate performance.ResultsFor cases and effective R_t, all three time horizons outperform the three credible levels of the forecast. For hospitalizations, all three time horizons outperform the 20% and 50% credible intervals of the forecast. On the contrary, the 1-day and 3-day periods underperform the 90% credible intervals. Questions about uncertainty quantification should be re-calculated using the frequentist coverage probability of the Bayesian credible interval based on observed data for all three metrics.ConclusionsWe present an approach to automate the real-time estimation and forecasting of cases and hospitalizations and corresponding uncertainty using publicly available data. The models were able to infer short-term trends consistent with reported values at the HERC region level. Additionally, the models were able to accurately forecast and estimate the uncertainty of the measurements. This study can help identify the most affected regions and major outbreaks in the near future. The workflow can be adapted to other geographic regions, states, and even countries where decision-making processes are supported in real-time by the proposed modeling system.

Objective Bayesian analysis of the ratio of bivariate lognormal means

In this study, we consider the objective Bayesian analysis for the ratio of means in the bivariate lognormal distribution, in which properly assigning priors for the ratio of means is challenging due to the presence of nuisance parameters. As a result, we create probability matching priors and reference priors for the ratio of means. Jeffreys’ prior and the two group reference prior do not satisfy a first-order matching criterion. We also investigate the posterior distribution property for the general prior class, which includes Jeffreys’ prior, the reference prior, and the matching prior. Through a simulation study, we check the frequentist coverage probabilities and compare them with the generalized confidence interval method. It demonstrated that the proposed probability matching priors match very well with the target coverage probabilities even when the sample sizes are small in a frequentist sense. Two real examples are also provided.

Bayesian Inference of System Reliability for Multicomponent Stress-Strength Model under Marshall-Olkin Weibull Distribution

Industrial systems often have redundant structures for improving reliability and avoiding sudden failures, and a parallel system is one of the special redundant systems. In this paper, we consider the problem of reliability estimation for a parallel system when one stress variable is involved, which is called the multicomponent stress-strength model. The parallel system contains two components, and their joint lifetime follows a Marshall–Olkin bivariate Weibull distribution, while the stress variable is assumed to be the Weibull distribution. Due to the complicated form of the likelihood function, a data augmentation method is proposed, and then the Gibbs sampling algorithm is constructed to obtain the Bayesian estimation of the system reliability. The proposed method is evaluated by a simulated dataset and Monte Carlo simulation study. The simulation results show that the proposed method performs well in terms of relative bias, mean squared error and frequentist coverage probability.

Matching Bayesian and frequentist coverage probabilities when using an approximate data covariance matrix

ABSTRACT Observational astrophysics consists of making inferences about the Universe by comparing data and models. The credible intervals placed on model parameters are often as important as the maximum a posteriori probability values, as the intervals indicate concordance or discordance between models and with measurements from other data. Intermediate statistics (e.g. the power spectrum) are usually measured and inferences are made by fitting models to these rather than the raw data, assuming that the likelihood for these statistics has multivariate Gaussian form. The covariance matrix used to calculate the likelihood is often estimated from simulations, such that it is itself a random variable. This is a standard problem in Bayesian statistics, which requires a prior to be placed on the true model parameters and covariance matrix, influencing the joint posterior distribution. As an alternative to the commonly used independence Jeffreys prior, we introduce a prior that leads to a posterior that has approximately frequentist matching coverage. This is achieved by matching the covariance of the posterior to that of the distribution of true values of the parameters around the maximum likelihood values in repeated trials, under certain assumptions. Using this prior, credible intervals derived from a Bayesian analysis can be interpreted approximately as confidence intervals, containing the truth a certain proportion of the time for repeated trials. Linking frequentist and Bayesian approaches that have previously appeared in the astronomical literature, this offers a consistent and conservative approach for credible intervals quoted on model parameters for problems where the covariance matrix is itself an estimate.

Objective priors for common correlation coefficient in bivariate normal populations

Various objective priors have been defined for the common correlation coefficient concerning several bivariate normal populations. In this paper, the proposed approach relies on the asymptotic matching of coverage probabilities corresponding to Bayesian credible intervals considering the corresponding frequentist ones. In the present paper, we focus on several matching criteria including quantile matching, distribution function matching, highest posterior density matching, and matching via inversion of test statistics. In addition, we consider reference priors for different groups of ordering. The proposed methods are investigated and compared between each other in terms of a frequentist coverage probability and then, they are illustrated through a simulation study and two real data examples.

Model-free posterior inference on the area under the receiver operating characteristic curve

The area under the receiver operating characteristic curve (AUC) serves as a summary of a binary classifier’s performance. For inference on the AUC, a common modeling assumption is binormality, which restricts the distribution of the score produced by the classifier. However, this assumption introduces an infinite-dimensional nuisance parameter and may be restrictive in certain machine learning settings. To avoid making distributional assumptions, and to avoid the computational challenges of a fully nonparametric analysis, we develop a direct and model-free Gibbs posterior distribution for inference on the AUC. We present the asymptotic Gibbs posterior concentration rate, and a strategy for tuning the learning rate so that the corresponding credible intervals achieve the nominal frequentist coverage probability. Simulation experiments and a real data analysis demonstrate the Gibbs posterior’s strong performance compared to existing Bayesian methods.

Empirical Priors and Coverage of Posterior Credible Sets in a Sparse Normal Mean Model

Bayesian methods provide a natural means for uncertainty quantification, that is, credible sets can be easily obtained from the posterior distribution. But is this uncertainty quantification valid in the sense that the posterior credible sets attain the nominal frequentist coverage probability? This paper investigates the frequentist validity of posterior uncertainty quantification based on a class of empirical priors in the sparse normal mean model. In particular, we show that our marginal posterior credible intervals achieve the nominal frequentist coverage probability under conditions slightly weaker than needed for selection consistency and a Bernstein--von Mises theorem for the full posterior, and numerical investigations suggest that our empirical Bayes method has superior frequentist coverage probability properties compared to other fully Bayes methods.

Calibrating general posterior credible regions

Summary Calibration of credible regions derived from under- or misspecified models is an important and challenging problem. In this paper, we introduce a scalar tuning parameter that controls the posterior distribution spread, and develop a Monte Carlo algorithm that sets this parameter so that the corresponding credible region achieves the nominal frequentist coverage probability.

Interval estimation for Wiener processes based on accelerated degradation test data

Degradation is a primary cause of failures for many materials and products. Although stochastic processes have been widely applied to degradation data, there is a lack of efficient and accurate methods for interval estimation of model parameters and reliability characteristics given limited degradation data. Using the method of generalized pivotal quantities, this study develops interval estimation procedures for fixed-effects and mixed-effects Wiener degradation processes based on accelerated degradation test data. The fixed-effects processes are common for mature products and the mixed-effects model is capable of capturing heterogeneities in an immature product. The constructed confidence intervals are shown to have exact, or asymptotically exact, frequentist coverage probabilities. Extensive simulations are conducted to compare the proposed procedures to other competing methods, including the large sample normal approximation, and the bootstrap. The simulation results reveal that the proposed intervals have satisfactory performance in terms of the coverage probability and the average interval length. The proposed interval estimation procedures are successfully applied to accelerated degradation data from commercial white LEDs.

Objective Bayesian analysis for the accelerated degradation model based on the inverse Gaussian process

The inverse Gaussian process is an attractive stochastic process to model monotone degradation paths in degradation analysis. In this paper, we propose an objective Bayesian method to analyze the accelerated degradation model based on the inverse Gaussian process. Noninformative priors including the Jeffreys prior and reference priors are derived, and the propriety of the posteriors under each prior is validated. A simulation study is carried out to investigate the performance of the approach compared with the maximum likelihood method and the Bootstrap method. Numerical results show that the proposed method has better performance in terms of the mean squared error and the frequentist coverage probability. The reference prior πR2 is recommended to use in practice. Finally, the Bayesian approach is applied to a real data.

A matching prior for the product of normal means based on the modified profile likelihood

ABSTRACTIn this paper, we develop a matching prior for the product of means in several normal distributions with unrestricted means and unknown variances. For this problem, properly assigning priors for the product of normal means has been issued because of the presence of nuisance parameters. Matching priors, which are priors matching the posterior probabilities of certain regions with their frequentist coverage probabilities, are commonly used but difficult to derive in this problem. We developed the first order probability matching priors for this problem; however, the developed matching priors are unproper. Thus, we apply an alternative method and derive a matching prior based on a modification of the profile likelihood. Simulation studies show that the derived matching prior performs better than the uniform prior and Jeffreys’ prior in meeting the target coverage probabilities, and meets well the target coverage probabilities even for the small sample sizes. In addition, to evaluate the validity of the proposed matching prior, Bayesian credible interval for the product of normal means using the matching prior is compared to Bayesian credible intervals using the uniform prior and Jeffrey’s prior, and the confidence interval using the method of Yfantis and Flatman.

A matching prior based on the modified profile likelihood for the common mean in multiple log-normal distributions

In this paper, we develop a matching prior for the common mean in several log-normal distributions. For this problem, assigning priors appropriately for the common log-normal mean is challenging owing to the presence of nuisance parameters. Matching priors, which are priors that match the posterior probabilities of certain regions within their frequentist coverage probabilities, are commonly used in this problem. However, a closed form posterior under the derived first order matching prior is not available; further, the second order matching prior is difficult to be derived in this problem. Thus, alternatively, we derive a matching prior based on a modification of the profile likelihood. Simulation studies show that this proposed prior meets the target coverage probabilities very well even for small sample sizes. Finally, we present a real example.

Objective Bayes models for compatibility assessment and bias estimation

The paper derives a class of non-informative, probability matching priors and of default, data-dependent priors for the difference in two normal means when the variance(s) are unknown. These distributions are useful to get a Bayesian model in the situation where one has to assess compatibility of data obtained by a laboratory with the specified values and uncertainty estimate given in the certificate of analysis. The latter is then interpreted as the parameter assignment for measurand’s normal distribution. The Bayes rules arising from these prior distributions are compared via the frequentist coverage probability of corresponding credible intervals for lab’s bias.

Objective Bayesian inference for the ratio of the scale parameters of two Weibull distributions

ABSTRACTThe Weibull distribution is widely used due to its versatility and relative simplicity. In our paper, the non informative priors for the ratio of the scale parameters of two Weibull models are provided. The asymptotic matching of coverage probabilities of Bayesian credible intervals is considered, with the corresponding frequentist coverage probabilities. We developed the various priors for the ratio of two scale parameters using the following matching criteria: quantile matching, matching of distribution function, highest posterior density matching, and inversion of test statistics. One particular prior, which meets all the matching criteria, is found. Next, we derive the reference priors for groups of ordering. We see that all the reference priors satisfy a first-order matching criterion and that the one-at-a-time reference prior is a second-order matching prior. A simulation study is performed and an example given.

Permissible noninformative priors for the accelerated life test model with censored data

In this paper, the Jeffreys priors for the step-stress partially accelerated life test with Type-II adaptive progressive hybrid censoring scheme data are considered. Given a density function family satisfied certain regularity conditions, the Fisher information matrix and Jeffreys priors are obtained. Taking the Weibull distribution as an example, the Jeffreys priors, posterior analysis and its permissibility are discussed. The results, which present that how the accelerated stress levels, censored size, hybrid censoring time and stress change time etc. affect the Jeffreys priors, are obtained. In addition, a theorem which shows there exists a relationship between single observation and multi observations for permissible priors is proved. Finally, using Metroplis with in Gibbs sampling algorithm, these factors are confirmed by computing the frequentist coverage probabilities.

Noninformative priors for the ratio of the shape parameters of two Weibull distributions

The Weibull distribution is one of the most widely used lifetime distributions in reliability engineering. Here, the noninformative priors for the ratio of the shape parameters of two Weibull models are introduced. The first criterion used is the asymptotic matching of the coverage probabilities of Bayesian credible intervals with the corresponding frequentist coverage probabilities. We develop the probability matching priors for the ratio of the shape parameters using the following matching criteria: quantile matching, matching of the distribution function, highest posterior density matching, and matching via inversion of the test statistics. We obtain one particular prior that meets all the matching criteria. Next, we derive the reference priors for different groups of ordering. Our findings show that some of the reference priors satisfy a first-order matching criterion and the one-at-a-time reference prior is a second-order matching prior. Lastly, we perform a simulation study and provide a real-world example.

Noninformative priors for the common shape parameters of Weibull distributions

The Weibull distribution is widely used in survival analysis, reliability engineering, and failure analysis. In this paper, we develop noninformative priors such as probability matching priors for the common shape parameter of Weibull models using the asymptotic matching of coverage probabilities of Bayesian credible intervals with the corresponding frequentist coverage probabilities. Here, various matching criteria, such as quantile matching, matching of distribution function, highest posterior density matching, and matching via inversion of test statistics, are considered. Next, we derive the reference priors for groups of ordering. It also shows that most reference priors do not satisfy the first order matching criterion, but the one-at-a-time reference prior is the second order matching priors. A simulation study is performed and a real example is given.

A New lifetime model for multivariate survival data with a surviving fraction

In this paper we propose a new lifetime model for multivariate survival data with a surviving fraction. We develop this model assuming that there are m types of unobservable competing risks, where each risk is related to a time of the occurrence of an event of interest. We explore the use of Markov chain Monte Carlo methods to develop a Bayesian analysis for the proposed model. We also perform a simulation study in order to analyse the frequentist coverage probabilities of credible interval derived from posteriors. Our modelling is illustrated through a real data set.

Objective Bayes analysis of zero-inflated Poisson distribution with application to healthcare data

In this article, non-informative priors are investigated for a zero-inflated Poisson distribution with two parameters: the probability of zeros and the mean of the Poisson part. Both the reference prior and the Jeffreys prior are derived and shown to be second-order matching priors when only the mean of the Poisson part is of interest. However, when the probability of zeros is of interest, the reference prior is still a second-order matching prior, whereas the Jeffreys prior is not so. Furthermore, when both parameters are of interest, the reference prior is a unique second-order matching prior. Frequentist coverage probabilities of the posterior confidence sets based on the Jeffreys and reference priors are compared with each other using Monte Carlo simulations and with confidence sets based on the maximum likelihood estimation.