Frequentist Confidence Intervals Research Articles

Bayesian and frequentist confidence intervals arising from empirical-type likelihoods

SUMMARY For a general class of empirical-type likelihoods for the population mean, higher-order asymp totics are developed with a view to characterizing its members which allow, for any given prior, the existence of a confidence interval that has approximately correct posterior as well as fre quentist coverage. In particular, it is seen that the usual empirical likelihood always allows such a confidence interval, while many of its variants proposed in the literature do not enjoy this property. An explicit form of the confidence interval is also given.

Confidence intervals based on empirical statistics: existence of a probability matching prior and connection with frequentist Bartlett adjustability

We consider frequentist confidence intervals based on the inversion of likelihood ratio statistics that arise from a very general class of empirical-type likelihoods. Higher order asymptotics for the posterior coverage of such confidence intervals are derived for the purpose of characterizing the members of the class that allow the existence of a probability matching prior. The connection with frequentist Bartlett adjustability is also investigated.

Heuristic Thinking and Inference From Observational Epidemiology

Epidemiologic research is an exercise in measurement. Observational epidemiologic results usually include a point estimate, a measure of random error such as a frequentist confidence interval, and a qualitative discussion of study limitations. Without randomization of study subjects to exposure groups, inference from study results requires an educated guess about the strength of the systematic errors compared with the strength of the exposure effects. Although quantitative methods to make these educated guesses exist, the conventional approach is qualitative, which reduces the educated guessing to a problem of reasoning under uncertainty. In circumstances such as these, humans predictably reason poorly. Heuristics and resulting biases that simplify the judgmental tasks tend to underestimate the systematic error, underestimate the uncertainty, and focus the inference on the study's specific evidence while excluding countervailing external information. Common warnings to interpret results with trepidation are an ineffective solution. The methods that quantify systematic error and uncertainty challenge the analyst to specify the alternative explanations for associations that are otherwise too readily judged causal.

Intrinsic credible regions: An objective Bayesian approach to interval estimation

This paper definesintrinsic credible regions, a method to produce objective Bayesian credible regions which only depends on the assumed model and the available data.Lowest posterior loss (LPL) regions are defined as Bayesian credible regions which contain values of minimum posterior expected loss: they depend both on the loss function and on the prior specification. An invariant, information-theory based loss function, theintrinsic discrepancy is argued to be appropriate for scientific communication. Intrinsic credible regions are the lowest posterior loss regions with respect to the intrinsic discrepancy loss and the appropriate reference prior. The proposed procedure is completely general, and it is invariant under both reparametrization and marginalization. The exact derivation of intrinsic credible regions often requires numerical integration, but good analytical approximations are provided. Special attention is given to one-dimensional intrinsic credible intervals; their coverage properties show that they are always approximate (and sometimes exact) frequentist confidence intervals. The method is illustrated with a number of examples.

Primordial non-Gaussianity: local curvature method and statistical significance of constraints onfNLfromWMAPdata

We test the consistency of estimates of the non-linear coupling constant f$_{NL$ using non-Gaussian cosmic microwave background (CMB) maps generated by the method described in the work of Liguori, Matarrese \amp Moscardini. This procedure to obtain non-Gaussian maps differs significantly from the method used in previous works on the estimation of f$_NL$. Nevertheless, using spherical wavelets, we find results in very good agreement with Mukherjee \amp Wang, showing that the two ways of generating primordial non-Gaussian maps give equivalent results. Moreover, we introduce a new method for estimating the non-linear coupling constant from CMB observations by using the local curvature of the temperature fluctuation field. We present both Bayesian credible regions (assuming a flat prior) and proper (frequentist) confidence intervals on f$_NL$, and discuss the relation between the two approaches. The Bayesian approach tends to yield lower error bars than the frequentist approach, suggesting that a careful analysis of the different interpretations is needed. Using this method, we estimate f$_NL$=-10$^+270$$_-260$ at the 2$\sigma$ level (Bayesian) and f$_NL$=-10$^+310$$_-270$ (frequentist). Moreover, we find that the wavelet and the local curvature approaches, which provide similar error bars, yield approximately uncorrelated estimates of f$_NL$ and therefore, as advocated in the work of Cabella et al., the estimates may be combined to reduce the error bars. In this way, we obtain f$_NL$=-5 +/- 85 and f$_NL$=-5 +/- 175 at the 1$\sigma$ and 2$\sigma$ level respectively using the frequentist approach. }

A sensitivity analysis of a randomized controlled trial of zinc in treatment of falciparum malaria in children

Background The randomized trial has long been recognized as a means to assess the efficacy of new interventions, because the investigator can reduce or eliminate many sources of error. As such, clinical trials often do not include quantitative assessments of the extent that systematic error could affect their results. We examined the impact of different sources of bias on a randomized controlled trial of the efficacy of zinc as an adjuvant to malaria therapy in reducing time to total parasite clearance. Methods Using data from a previously published study, we identified two sources of bias and used the sensitivity analysis technique developed by Lash and Fink to assess the impact of each source of bias on the outcome. Results After correcting for each source of bias and reincorporating random error into our results, the point estimate of effect comparing those who received placebo to those who received zinc changed slightly (SMR changed from 0.92 to 0.90) but the 95% interval increased 22% (changing from 0.73–1.16 in the conventional analysis to 0.65–1.26 in the sensitivity analysis). Conclusions The findings of this sensitivity analysis serve as a reminder that the frequentist confidence interval underestimates the total error, even in a randomized controlled trial. Authors of randomized controlled trial investigations ought to conduct a complete assessment of the impact of potential sources of bias in their studies. CONSORT guidelines for reporting trial results should be updated to encourage authors to assess the impact of non-random errors on their studies.

Bayesian inference on the patient population size given list mismatches

In applying capture-recapture methods for closed populations to epidemiology, one needs to estimate the total number of people with a certain disease in a certain research area by using several lists with information of patients. Problems of lists error often arise due to mistyping or misinformation. Adopting the concept of tag-loss methodology in animal populations, Seber et al. (Biometrics 2000; 56:1227-1232) proposed solutions to a two-list problem. This article reports an interesting simulation study, where Bayesian point estimates based on improper constant and Jeffreys prior for unknown population size N could have smaller frequentist standard errors and MSEs compared to the estimates proposed in Seber et al. (2000). The Bayesian credible intervals based on the same priors also have super frequentist coverage probabilities while some of the frequentist confidence intervals procedures have drastically poor coverage. Seber's real data set on gestational diabetics is analysed with the proposed new methods.

A program for confidence interval calculations for a Poisson process with background including systematic uncertainties: POLE 1.0

A Fortran 77 routine has been developed to calculate confidence intervals with and without systematic uncertainties using a frequentist confidence interval construction with a Bayesian treatment of the systematic uncertainties. The routine can account for systematic uncertainties in the background prediction and signal/background efficiencies. The uncertainties may be separately parametrized by a Gauss, log-normal or flat probability density function (PDF), though since a Monte Carlo approach is chosen to perform the necessary integrals a generalization to other parameterizations is particularly simple. Full correlation between signal and background efficiency is optional. The ordering schemes for frequentist construction currently supported are the likelihood ratio ordering (also known as Feldman–Cousins) and Neyman ordering. Optionally, both schemes can be used with conditioning, meaning the probability density function is conditioned on the fact that the actual outcome of the background process can not have been larger than the number of observed events. Program summary Title of program: POLE version 1.0 Catalogue identifier: ADTA Program summary URL: http://cpc.cs.qub.ac.uk/summaries/ADTA Program available from: CPC Program Library, Queen's University of Belfast, N. Ireland Licensing provisions: None Computer for which the program is designed: DELL PC 1 GB 2.0 Ghz Pentium IV Operating system under which the program has been tested: RH Linux 7.2 Kernel 2.4.7-10 Programming language used: Fortran 77 Memory required to execute with typical data: ∼1.6 Mbytes No. of bytes in distributed program, including test data, etc.: 373745 No. of lines in distributed program, including test data, etc.: 2700 Distribution format: tar gzip file Keywords: Confidence interval calculation, Systematic uncertainties Nature of the physical problem: The problem is to calculate a frequentist confidence interval on the parameter of a Poisson process with known background in presence of systematic uncertainties in experimental parameters such as signal efficiency or background prediction. Method of solution: A PDF for the confidence interval construction is constructed by folding a Poisson density function with a parametrization describing the posterior PDF of the systematic uncertainties. The construction is then performed with this new PDF according to different ordering schemes. The convolution is performed using a Monte Carlo integration method, providing greatest possible flexibility with respect to the parametrization. Restriction on the complexity of the problem: No restrictions. Typical running time: for calculation of a single confidence interval with 1000 steps in signal hypothesis space the wall clock time on a DELL PC 1 GB 2.0 Ghz Pentium IV with 4 GB swap is ca. 500 seconds. The wall clock time scales roughly linear with the number of steps in signal space. Unusual features of the program: POLE uses routines from the CERNLIB library [ http://wwwasd.web.cern.ch/wwwasd/cernlib], which have to be linked in during compilation. It should be noted that the number of steps in signal space (maximal s required divided by the step-width) should not exceed 1000, since the corresponding arrays are predimensioned.

Frequentist and Bayesian confidence intervals

Frequentist (classical) and Bayesian approaches to the construction of confidence limits are compared. Various examples which illustrate specific problems are presented. The Likelihood Principle and the Stopping Rule Paradox are discussed. The performance of the different methods is investigated relative to the properties coherence, precision, bias, universality, simplicity. A proposal on how to define error limits in various cases are derived from the comparison. They are based on the likelihood function only and follow in most cases the general practice in high energy physics. Classical methods are not recommended because they violate the Likelihood Principle, they can produce inconsistent results, suffer from lack of precision and generality. Also the extreme Bayesian approach with arbitrary choice of the prior probability density or priors deduced from scaling laws is rejected.

Frequentist estimation of cosmological parameters from the MAXIMA-1 cosmic microwave background anisotropy data

We use a frequentist statistical approach to set confidence intervals on the values of cosmological parameters using the MAXIMA-1 and COBE measurements of the angular power spectrum of the cosmic microwave background. We define a Δχ2 statistic, simulate the measurements of MAXIMA-1 and COBE, determine the probability distribution of the statistic, and use it and the data to set confidence intervals on several cosmological parameters. We compare the frequentist confidence intervals with Bayesian credible regions. The frequentist and Bayesian approaches give best estimates for the parameters that agree within 15 per cent, and confidence interval widths that agree to within 30 per cent. The results also suggest that a frequentist analysis gives slightly broader confidence intervals than a Bayesian analysis. The frequentist analysis gives values of , and , and the Bayesian analysis gives values of , and , all at the 95 per cent confidence level.

The statistical analysis of Gaussian and Poisson signals near physical boundaries

We propose a construction of frequentist confidence intervals that is effective near unphysical regions and unifies the treatment of two-sided and upper limit intervals. It is rigorous, has coverage, is computationally simple and avoids the pathologies that affect the likelihood ratio and related constructions. Away from nonphysical regions, the results are exactly the usual central two-sided intervals. The construction is based on including the physical constraint in the derivation of the estimator, leading to an estimator with values that are confined to the physical domain.

Sequential sampling of normal and non-normal populations

This paper reports a simulation-based exploration into the computation of point and interval estimates for data arising from sequential sampling of populations with a variety of underlying distributions. We conclude that the coverage probability of the standard frequentist confidence interval estimates is overstated. The effect of non-normal behavior in the underlying population upon the properties of the interval estimate varies, depending upon the severity and type of anomaly (skewness, kurtosis, etc). Sequential sampling of certain nonsymmetric distributions performed very poorly compared with simple random sampling. We assess other interval estimates that do not overstate the coverage probability if the underlying population is normal (Gaussian).

Comment: The First Data Analysis Should be Journalistic

Bayesian statistical methods can be considered an attempt at mathematical formalization of the natural scientific process of interpretation of data in light of preexisting information. As such, their use, and the degree to which they are used, is largely a question of efficiency. In some instances it may be appropriate to incorporate prior information into an analysis to the extent that this information is deemed reliable by all concerned; it will not often be the case in an ecological study, however, that information satisfying these constraints is substantial. In the most important distributional setting a frequentist confidence interval is identical to a noninformative Bayesian credible interval, and it is asserted that in most other cases where noninformative priors are used, these two will be very similar; the primary data analysis in an ecological study should probably be of one of these two forms. It is conjectured that, in order to be mathematically tractable, decision theoretic methods (Bayesian or not) will often deal with a dangerously short action‐space time frame. Finally, Empirical Bayesian methods and hierarchical models in general are powerful new methods that should be used, with caution, to the extent that their superstructural assumptions are reliable.

Helping doctors to draw appropriate inferences from the analysis of medical studies.

Most clinicians and many medical statisticians interpret standard frequentist confidence intervals by invoking the Bayesian concept of subjective probability. Fortunately, the assumptions that render this interpretation acceptable are often quite reasonable in the setting of the practical day-to-day analysis of medical data. This article takes the subjective interpretation of confidence intervals to its logical conclusion and argues that the inferential understanding of clinicians and public health physicians could potentially be improved if, where it was appropriate, standard inferential statements--point estimates, 95 per cent confidence intervals and P-values--were supplemented by estimates of the subjective posterior probability, assuming a uniform prior density, that the true value of a parameter to be estimated exceeds one or a series of thresholds that are clinically critical or easily interpretable. Many decision makers in the health care arena draw totally inappropriate inferences from analyses where the point estimate indicates a clinically valuable effect but the null hypothesis cannot formally be rejected, and, although the proposed approach could be of potential value in a range of settings, it is argued that it could be of particular use in the rational interpretation of underpowered studies that must inform critical clinical or public health decisions.

Estimation issues in clinical trials and overviews

There is a general move towards greater emphasis on point and interval estimates of treatment effect in reporting of clinical trials, so that significance testing plays a lesser role. In this article we examine a number of issues which affect the use and interpretation of conventional estimation methods. Should we accept or avoid the stereotypes of 95 per cent confidence? Should the abstract of a trial report include confidence intervals for major endpoints? Are frequentist confidence intervals being interpreted correctly, and should Bayesian probability intervals be more widely used in trial reports? Does the timing of publication, such as early stopping because of a large observed treatment difference, lead to exaggerated point and interval estimates? How can we produce realistic estimates from subgroup analyses? Is publication bias seriously affecting our ability to obtain unbiased estimates? Is the emphasis on estimation methods a powerful tool for encouraging larger sample sizes? Can we resolve the controversy concerning fixed or random effects models for estimation in overviews of related trials? Our arguments are illustrated by results from recent trials in cardiovascular disease.

Correct Bayesian and frequentist intervals are similar

This paper argues that Bayesians and frequentists will normally reach numerically similar conclusions, when dealing with vague data or sparse data. It is shown that both statistical methodologies can deal reasonably with vague data. With sparse data, in many important practical cases Bayesian interval estimates and frequentist confidence intervals are approximately equal, although with discrete data the frequentist intervals are somewhat longer. This is not to say that the two methodologies are equally easy to use: The construction of a frequentist confidence interval may require new theoretical development. Bayesian methods typically require numerical integration, perhaps over many variables. Also, Bayesians can easily fall into the trap of over-optimism about their amount of prior knowledge. But in cases where both intervals are found correctly, the two intervals are usually not very different.