Frequentist Coverage Probabilities Research Articles

Objective Bayesian analysis for autoregressive models with nugget effects

The conditional autoregressive (CAR) and simultaneous autoregressive (SAR) models both have been used extensively for the analysis of spatial structure underlying lattice data in many areas, such as epidemiology, demographics, economics, and geography. Default Bayesian analyses have been conducted recently, but the Bayesian approach has not used or explored these two models with nugget effects. In this paper, we consider general autoregressive models including both CAR and SAR models. The Jeffreys-rule, independence Jeffreys, commonly used reference and “exact” reference priors are derived. The propriety of the marginal priors and joint posteriors is studied for a large class of objective priors. Various Jeffreys and reference priors are shown to yield improper posteriors and only the Jeffreys-rule and the “exact” reference priors yield proper posteriors. We make comparisons for these two objective priors using the frequentist coverage probabilities of the credible intervals. An illustration is given using a real spatial data-set.

Empirical Bayes Confidence Intervals for Selected Parameters in High-Dimensional Data

Modern statistical problems often involve a large number of populations and hence a large number of parameters that characterize these populations. It is common for scientists to use data to select the most significant populations, such as those with the largest t statistics. The scientific interest often lies in studying and making inferences regarding these parameters, called the selected parameters, corresponding to the selected populations. The current statistical practices either apply a traditional procedure assuming there were no selection—a practice that is not valid—or they use the Bonferroni-type procedure that is valid but very conservative and often noninformative. In this article, we propose valid and sharp confidence intervals that allow scientists to select parameters and to make inferences for the selected parameters based on the same data. This type of confidence interval allows the users to zero in on the most interesting selected parameters without collecting more data. The validity of confidence intervals is defined as the controlling of Bayes coverage probability so that it is no less than a nominal level uniformly over a class of prior distributions for the parameter. When a mixed model is assumed and the random effects are the key parameters, this validity criterion is exactly the frequentist criterion, since the Bayes coverage probability is identical to the frequentist coverage probability. Assuming that the observations are normally distributed with unequal and unknown variances, we select parameters with the largest t statistics. We then construct sharp empirical Bayes confidence intervals for these selected parameters, which have either a large Bayes coverage probability or a small Bayes false coverage rate uniformly for a class of priors. Our intervals, applicable to any high-dimensional data, are applied to microarray data and are shown to be better than all the alternatives. It is also anticipated that the same intervals would be valid for any selection rule. Supplementary materials for this article are available online.

Objective Bayesian analysis for bivariate Marshall–Olkin exponential distribution

The Bayesian estimators for the unknown parameters of the bivariate Marshall–Olkin exponential distribution under noninformative priors have been considered and several reference priors have been derived. A class of priors is found by matching the coverage probability of one-side Bayesian credible intervals with the corresponding frequentist coverage probabilities. It is noted that some of the reference priors are also matching priors and the posterior distributions based on the reference priors and matching priors are proper. Closed forms of Bayesian estimators are obtained with respect to the quadratic loss function. Gibbs sampling is utilized to obtain the credible intervals and coverage probabilities of parameters. Comparisons in the efficiency of the maximum likelihood estimators and Bayesian estimators under different reference priors and matching priors for various sample sizes have been done by Monte Carlo simulations. A real data set is analyzed for illustrative purpose.

On Bayesian credible sets, restricted parameter spaces and frequentist coverage

For estimating a lower bounded parametric function in the framework of Marchand and Strawderman [6], we provide through a unified approach a class of Bayesian confidence intervals with credibility $1-\alpha$ and frequentist coverage probability bounded below by $\frac{1-\alpha}{1+\alpha}$. In cases where the underlying pivotal distribution is symmetric, the findings represent extensions with respect to the specification of the credible set achieved through the choice of a spending function, and include Marchand and Strawderman’s HPD procedure result. For non-symmetric cases, the determination of a such a class of Bayesian credible sets fills a gap in the literature and includes an “equal-tails” modification of the HPD procedure. Several examples are presented demonstrating wide applicability.

A matching prior based on the modified profile likelihood in a generalized Weibull stress‐strength model

AbstractBayesian inference of a generalized Weibull stress‐strength model (SSM) with more than one strength component is considered. For this problem, properly assigning priors for the reliabilities is challenging due to the presence of nuisance parameters. Matching priors, which are priors matching the posterior probabilities of certain regions with their frequentist coverage probabilities, are commonly used but difficult to derive in this problem. Instead, we apply an alternative method and derive a matching prior based on a modification of the profile likelihood. Simulation studies show that this proposed prior performs well in terms of frequentist coverage and estimation even when the sample sizes are minimal. The prior is applied to two real datasets. The Canadian Journal of Statistics 41: 83–97; 2013 © 2012 Statistical Society of Canada

Objective Bayesian analysis for CAR models

Objective priors, especially reference priors, have been studied extensively for spatial data in the last decade. In this paper, we study objective priors for a CAR model. In particular, the properties of the reference prior and the corresponding posterior are studied. Furthermore, we show that the frequentist coverage probabilities of posterior credible intervals depend only on the spatial dependence parameter \(\rho \), and not on the regression coefficient or the error variance. Based on the simulation study for comparing the reference and Jeffreys priors, the performance of two reference priors is similar and better than the Jeffreys priors. One spatial dataset is used for illustration.

Bayes Inference for a Tractable New Class of Non-symmetric Distributions for 3-Dimensional Rotations

Both existing models for non-symmetric distributions on 3-dimensional rotations and their associated one-sample inference methods have serious limitations in terms of both interpretability and ease of use. Based on the intuitively appealing Uniform Axis- Random Spin (UARS) construction of Bingham, Nordman, and Vardeman (2009) for symmetric families of distributions, we propose new highly interpretable and tractable classes of non-symmetric distributions that are derived from mixing UARS distributions. These have an appealing Preferred Axis-Random Spin (PARS) construction and (unlike existing models) directly interpretable parameters. Non-informative one-sample Bayes inference in these models is a direct generalization of UARS methods introduced in Bingham, Vardeman, and Nordman (2009), where credible levels were found to be essentially equivalent to frequentist coverage probabilities. We apply the new models and inference methods to a problem in biomechanics, where comparison of model parameters provides meaningful comparisons for the nature of movement about the calcaneocuboid joint of three different primate subjects.

Coverage Properties of Confidence Intervals for Generalized Additive Model Components

Abstract. We study the coverage properties of Bayesian confidence intervals for the smooth component functions of generalized additive models (GAMs) represented using any penalized regression spline approach. The intervals are the usual generalization of the intervals first proposed by Wahba and Silverman in 1983 and 1985, respectively, to the GAM component context. We present simulation evidence showing these intervals have close to nominal ‘across‐the‐function’ frequentist coverage probabilities, except when the truth is close to a straight line/plane function. We extend the argument introduced byNychka in 1988for univariate smoothing splines to explain these results. The theoretical argument suggests that close to nominal coverage probabilities can be achieved, provided that heavy oversmoothing is avoided, so that the bias is not too large a proportion of the sampling variability. The theoretical results allow us to derive alternative intervals from a purely frequentist point of view, and to explain the impact that the neglect of smoothing parameter variability has on confidence interval performance. They also suggest switching the target of inference for component‐wise intervals away from smooth components in the space of the GAM identifiability constraints.

Non-informative priors in the generalized gamma stress–strength systems

This article deals with non-informative priors for parameters when both stress and strength follow generalized gamma distributions. First, the orthogonal reparameterization is treated and then, using this reparameterization, Jeffreys’ prior, group ordering reference priors, and matching priors are derived. The propriety of posterior distributions is investigated and marginal posterior distributions are provided under those non-informative priors. The question of whether or not the reference priors satisfy the probability matching criterion is addressed. Finally, the reference prior that satisfies the probability matching criterion is shown to be good in the sense of frequentist coverage probability of the posterior quantile. Accepted in 2005 for a special issue on Reliability co-edited by Hoang Pham, Rutgers University; Dong Ho Park, Hallym University, Korea; and Richard Cassady, University of Arkansas.

The multivariate Behrens–Fisher distribution

The main purpose of this paper is the study of the multivariate Behrens–Fisher distribution. It is defined as the convolution of two independent multivariate Student t distributions. Some representations of this distribution as the mixture of known distributions are shown. An important result presented in the paper is the elliptical condition of this distribution in the special case of proportional scale matrices of the Student t distributions in the defining convolution. For the bivariate Behrens–Fisher problem, the authors propose a non-informative prior distribution leading to highest posterior density (H.P.D.) regions for the difference of the mean vectors whose coverage probability matches the frequentist coverage probability more accurately than that obtained using the independence-Jeffreys prior distribution, even with small samples.

Frequentist coverage properties of uncertainty intervals for weak Poisson signals in the presence of background

We construct uncertainty intervals for weak Poisson signals in the presence of background. We consider the case where a primary experiment yields a realization of the signal plus background, and a second experiment yields a realization of the background. The data acquisition times, for the background-only experiment, Tbg, and the primary experiment, T, are selected so that their ratio, Tbg/T, varies from 1 to 25. The upper choice of 25 is motivated by an experimental study at the National Institute of Standards and Technology (NIST). The expected number of background counts in the primary experiment varies from 0.2 to 2. We construct 90% and 95% confidence intervals based on a propagation-of-errors method as well as two implementations of a Neyman procedure where acceptance regions are constructed based on a likelihood-ratio criterion that automatically determines whether the resulting confidence interval is one-sided or two-sided. In one of the implementations of the Neyman procedure due to Feldman and Cousins (FC), uncertainty in the expected background contribution is neglected. In the other implementation, we account for random uncertainty in the estimated expected background with a parametric bootstrap implementation of a method due to Conrad. We also construct minimum length Bayesian credibility intervals. For each method, we test for the presence of a signal based on the value of the lower endpoint of the uncertainty interval. In general, the propagation-of-errors method performs the worst compared to the other methods according to frequentist coverage and detection probability criteria, and sometimes produces nonsensical intervals where both endpoints are negative. The Neyman procedures generally yield intervals with better frequentist coverage properties compared to the Bayesian method except for some cases where Tbg/T = 1. In general, the Bayesian method yields intervals with lower detection probabilities compared to Neyman procedures. One of the main conclusions is that when Tbg/T is 5 or more and the expected background is 2 or less, the FC method outperforms the other methods considered. For Tbg/T = 1, 2 we observe that the Neyman procedure methods yield false detection probabilities for the case of no signal that are higher than expected given the nominal frequentist coverage of the interval. In contrast, for Tbg/T = 1, 2, the false detection probability of the Bayesian method is less than expected according to the nominal frequentist coverage.

Bayesian and likelihood-based inference for the bivariate normal correlation coefficient

The paper develops some objective priors for correlation coefficient of the bivariate normal distribution. The criterion used is the asymptotic matching of coverage probabilities of Bayesian credible intervals with the corresponding frequentist coverage probabilities. The paper uses various matching criteria, namely, quantile matching, highest posterior density matching, and matching via inversion of test statistics. Each matching criterion leads to a different prior for the parameter of interest. We evaluate their performance by comparing credible intervals through simulation studies. In addition, inference through several likelihood-based methods have been discussed.

Frequentist evaluation of intervals estimated for a binomial parameter and for the ratio of Poisson means

Confidence intervals for a binomial parameter or for the ratio of Poisson means are commonly desired in high energy physics (HEP) applications such as measuring a detection efficiency or branching ratio. Due to the discreteness of the data, in both of these problems the frequentist coverage probability unfortunately depends on the unknown parameter. Trade-offs among desiderata have led to numerous sets of intervals in the statistics literature, while in HEP one typically encounters only the classic intervals of Clopper–Pearson (central intervals with no undercoverage but substantial over-coverage) or a few approximate methods which perform rather poorly. If strict coverage is relaxed, some sort of averaging is needed to compare intervals. In most of the statistics literature, this averaging is over different values of the unknown parameter, which is conceptually problematic from the frequentist point of view in which the unknown parameter is typically fixed. In contrast, we perform an (unconditional) average over observed data in the ratio-of-Poisson-means problem. If strict conditional coverage is desired, we recommend Clopper–Pearson intervals and intervals from inverting the likelihood ratio test (for central and non-central intervals, respectively). Lancaster's mid- P modification to either provides excellent unconditional average coverage in the ratio-of-Poisson-means problem.

Fiducial intervals for the magnitude of a complex-valued quantity

This paper discusses a fiducial approach for constructing uncertainty intervals for the distance between k normal means and the origin. When k = 2 this distance is equivalent to the magnitude of a complex-valued quantity. A simulation study was conducted to assess the frequentist performance of the proposed fiducial intervals and to compare their performance with the methods from the Guide to the Expression of Uncertainty in Measurement and from Supplement 1 to the ‘Guide to the Expression of Uncertainty in Measurement’—Propagation of Distributions using a Monte Carlo Method. Our results indicate that the fiducial intervals generally outperform the GUM and GUM Supplement 1 methods with respect to frequentist coverage probabilities. Computer programs for calculating the fiducial intervals, written using open-source software, are listed.

Fiducial Intervals for Variance Components in an Unbalanced Two-Component Normal Mixed Linear Model

In this article we propose a new method for constructing confidence intervals for σα2,σϵ2, and the intraclass correlation ρ==σα2(σα2++σε2) in a two-component mixed-effects linear model. This method is based on an extension of R. A. Fisher's fiducial argument. We conducted a simulation study to compare the resulting interval estimates with other competing confidence interval procedures from the literature. Our results demonstrate that the proposed fiducial intervals have satisfactory performance in terms of coverage probability, as well as shorter average confidence interval lengths overall. We also prove that these fiducial intervals have asymptotically exact frequentist coverage probability. The computations for the proposed procedures are illustrated using real data examples.

A Modification for Bayesian Credible Intervals

In Bayesian analysis, people usually report the highest posterior density (HPD) credible interval as an interval estimate of an unknown parameter. However, when the unknown parameter is the nonnegative normal mean, the Bayesian HPD credible interval under the uniform prior has quite a low minimum frequentist coverage probability. To enhance the minimum frequentist coverage probability of a credible interval, I propose a new method of reporting the Bayesian credible interval. Numerical results show that the new reported credible interval has a much higher minimum frequentist coverage probability than the HPD credible interval.

On the implementation of local probability matching priors for interest parameters

Probability matching priors are priors for which the posterior probabilities of certain specified sets are exactly or approximately equal to their coverage probabilities. These priors arise as solutions of partial differential equations that may be difficult to solve, either analytically or numerically. Recently Levine & Casella (2003) presented an algorithm for the implementation of probability matching priors for an interest parameter in the presence of a single nuisance parameter. In this paper we develop a local implementation that is very much more easily computed. A local probability matching prior is a data-dependent approximation to a probability matching prior and is such that the asymptotic order of approximation of the frequentist coverage probability is not degraded. We illustrate the theory with a number of examples, including three discussed in Levine & Casella (2003).

Bayesian inference on the patient population size given list mismatches

In applying capture-recapture methods for closed populations to epidemiology, one needs to estimate the total number of people with a certain disease in a certain research area by using several lists with information of patients. Problems of lists error often arise due to mistyping or misinformation. Adopting the concept of tag-loss methodology in animal populations, Seber et al. (Biometrics 2000; 56:1227-1232) proposed solutions to a two-list problem. This article reports an interesting simulation study, where Bayesian point estimates based on improper constant and Jeffreys prior for unknown population size N could have smaller frequentist standard errors and MSEs compared to the estimates proposed in Seber et al. (2000). The Bayesian credible intervals based on the same priors also have super frequentist coverage probabilities while some of the frequentist confidence intervals procedures have drastically poor coverage. Seber's real data set on gestational diabetics is analysed with the proposed new methods.

Noninformative priors for the nested design

This paper considers noninformative priors for three-stage nested designs. It turns out that the noninformative prior given by Li and Stern (1997) is the one-at-a-time reference prior satisfying a second-order matching criterion when either the variance ratio or linear combinations of the means is of interest. Moreover, it is a joint probability matching prior when both the variance ratio and linear combinations of the means are of interest. These priors are compared with Jeffreys' prior in light of how accurately the coverage probabilities of Bayesian credible intervals match the corresponding frequentist coverage probabilities.

Interval estimation of genetic susceptibility for retrospective case-control studies

BackgroundThis article describes classical and Bayesian interval estimation of genetic susceptibility based on random samples with pre-specified numbers of unrelated cases and controls.ResultsFrequencies of genotypes in cases and controls can be estimated directly from retrospective case-control data. On the other hand, genetic susceptibility defined as the expected proportion of cases among individuals with a particular genotype depends on the population proportion of cases (prevalence). Given this design, prevalence is an external parameter and hence the susceptibility cannot be estimated based on only the observed data. Interval estimation of susceptibility that can incorporate uncertainty in prevalence values is explored from both classical and Bayesian perspective. Similarity between classical and Bayesian interval estimates in terms of frequentist coverage probabilities for this problem allows an appealing interpretation of classical intervals as bounds for genetic susceptibility. In addition, it is observed that both the asymptotic classical and Bayesian interval estimates have comparable average length. These interval estimates serve as a very good approximation to the "exact" (finite sample) Bayesian interval estimates. Extension from genotypic to allelic susceptibility intervals shows dependency on phenotype-induced deviations from Hardy-Weinberg equilibrium.ConclusionsThe suggested classical and Bayesian interval estimates appear to perform reasonably well. Generally, the use of exact Bayesian interval estimation method is recommended for genetic susceptibility, however the asymptotic classical and approximate Bayesian methods are adequate for sample sizes of at least 50 cases and controls.