Conservative Confidence Intervals Research Articles

A confidence region for the ridge path in multiple response surface optimization

Ridge analysis allows the analyst to explore the optimal operating conditions of the experimental factors. A confidence region is desirable for the estimated ridge path. Most literature concentrates on the univariate response situation. Little is known for the confidence region of the ridge path for the multivariate response; only a large-sample confidence interval for the ridge path is available. The simultaneous coverage rate for the existing interval is typically too conservative in practice, especially for small sample sizes. In this paper, the ridge path (via desirability function) is estimated based on the seemingly unrelated regression (SUR) model as well as standard multivariate regression (SMR) model, and a conservative confidence interval suitable for small sample sizes is proposed. It is shown that the proposed method outperforms the existing methods. Real-life examples and simulative study are given for illustration.

Confidence Regions and Intervals for Meta-Analysis Model Parameters

This article obtains confidence regions for the heteroscedastic, one-way random effects model’s parameters the heteroscedastic, one-way random effects model. The confidence regions are based on canonical representations of the restricted and profile likelihood functions in terms of independent normal random variables and χ2 random variables. These regions provide conservative confidence intervals for the common mean and heterogeneity variance. Mathematical details and the R code are available online as supplementary material.

Polarization measurement analysis

With the forthcoming release of high precision polarization measurements, such as from the Planck satellite, the metrology of polarization needs to be improved. In particular, it is important to have full knowledge of the noise properties when estimating polarization fraction and polarization angle, which suffer from well-known biases. While strong simplifying assumptions have usually been made in polarization analysis, we present a method for including the full covariance matrix of the Stokes parameters in estimates of the distributions of the polarization fraction and angle. We thereby quantified the impact of the noise properties on the biases in the observational quantities and derived analytical expressions for the probability density functions of these quantities that take the full complexity of the covariance matrix into account, including the Stokes I intensity components. We performed Monte Carlo simulations to explore the impact of the noise properties on the statistical variance and bias of the polarization fraction and angle. We show that for low variations (< 10%) of the effective ellipticity between the Q and U components around the symmetrical case the covariance matrix may be simplified as is usually done, with a negligible impact on the bias. For S/Ns with intensity lower than 10, the uncertainty on the total intensity is shown to drastically increase the uncertainty of the polarization fraction but not the relative bias of the polarization fraction, while a 10% correlation between the intensity and the polarized components does not significantly affect the bias of the polarization fraction. We compare estimates of the uncertainties that affect polarization measurements, addressing limitations of the estimates of the S/N, and we show how to build conservative confidence intervals for polarization fraction and angle simultaneously. This study, which is the first in a set of papers dedicated to analysing polarization measurements, focuses on the basic polarization fraction and angle measurements. It covers the noise regime where the complexity of the covariance matrix may be largely neglected in order to perform further analysis. A companion paper focuses on the best estimators of the polarization fraction and angle and on their associated uncertainties.

Sharp bounds on the variance in randomized experiments

We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials.

Estimation of Population Mean Under Unequal Probability Sampling with Unknown Selection Probabilities

In many practical problems of biology, medicine, and social sciences, inference about population mean has to be made froma sample generated with unknown selection probabilities that may vary between the units in the population. The most common approach to analysis of such samples is to use the ordinary sample mean as an estimator of population mean, under the assumption that the sample is “representative” of the population, although the concept of representative sampling is not given a formal mathematical definition. The main objective of this paper is to identify a necessary and sufficient condition for unbiasedness of the ordinary sample mean under unequal probability sampling. It is shown that in the process of sampling with unequal selection probabilities with a fixed sample size, expectation of the ordinary sample mean is exactly equal to population mean plus the covariance of the unit-level outcome with the unit-level relative selection probability, where the relative selection probability is defined as the ratio of the unit’s selection probability to the mean selection probability in the population. Hence, ordinary sample mean is unbiased if and only if the unit-level outcome is uncorrelated with the unit-level relative selection probability. Samples generated under this condition may be considered “representative” of the population for the purpose of inference about population mean. It is also shown that under representative sampling, asymptotically conservative confidence intervals for population mean can be constructed based on the ordinary sample variance estimator as long as there are no positive correlations among the sample observations (negative correlations are allowed) and conditions for asymptotic normality of the sample mean are satisfied.

Inference following designs which adjust for imbalances in prognostic factors

When the minimisation method due to Taves is used to balance treatment groups across prognostic factors, a problem arises at the time of analysing the results. Since minimisation is essentially a deterministic method, any statistical test based on the assumption of complete randomisation should not be used in the analysis. Previous articles have shown that analysis of covariance (ANOCOVA) produces valid tests. In this article, these results are extended to trials with more than one prognostic factor and more than two treatments. An alternative design to minimisation which makes use of optimum design theory is also considered, with two choices of biased coin. Simulation is used to study the effect on the power and the coverage probabilities of the usual tests and confidence intervals when these different allocation methods are applied. The results are then illustrated using data from an actual clinical trial. Simulation shows that when ANOCOVA is used, it is sometimes more powerful with these designs than with minimisation and produces slightly conservative confidence intervals for the treatment mean differences. The increase in power and conservativeness is more pronounced when there are more prognostic factors. The possibility of treatment-covariate interactions is also addressed. Results are only given when treatment responses are normally distributed. Under the simulated situations considered, when a covariate-adaptive design is used, the use of ANOCOVA yields a test which preserves the nominal significance level as compared to the conservativeness of analysis of variance.

Fast estimation of posterior probabilities in change-point analysis through a constrained hidden Markov model

The detection of change-points in heterogeneous sequences is a statistical challenge with applications across a wide variety of fields. In bioinformatics, a vast amount of methodology exists to identify an ideal set of change-points for detecting Copy Number Variation (CNV). While considerable efficient algorithms are currently available for finding the best segmentation of the data in CNV, relatively few approaches consider the important problem of assessing the uncertainty of the change-point location. Asymptotic and stochastic approaches exist but often require additional model assumptions to speed up the computations, while exact methods generally have quadratic complexity which may be intractable for large data sets of tens of thousands points or more. A hidden Markov model, with constraints specifically chosen to correspond to a segment-based change-point model, provides an exact method for obtaining the posterior distribution of change-points with linear complexity. The methods are implemented in the R package postCP, which uses the results of a given change-point detection algorithm to estimate the probability that each observation is a change-point. The results include an implementation of postCP on a publicly available CNV data set (n=120). Due to its frequentist framework, postCP obtains less conservative confidence intervals than previously published Bayesian methods, but with linear complexity instead of quadratic. Simulations showed that postCP provided comparable loss to a Bayesian MCMC method when estimating posterior means, specifically when assessing larger scale changes, while being more computationally efficient. On another high-resolution CNV data set (n=14,241), the implementation processed information in less than one second on a mid-range laptop computer.

An algorithm to compute the power of Monte Carlo tests with guaranteed precision

This article presents an algorithm that generates a conservative confidence interval of a specified length and coverage probability for the power of a Monte Carlo test (such as a bootstrap or permutation test). It is the first method that achieves this aim for almost any Monte Carlo test. Previous research has focused on obtaining as accurate a result as possible for a fixed computational effort, without providing a guaranteed precision in the above sense. The algorithm we propose does not have a fixed effort and runs until a confidence interval with a user-specified length and coverage probability can be constructed. We show that the expected effort required by the algorithm is finite in most cases of practical interest, including situations where the distribution of the $p$-value is absolutely continuous or discrete with finite support. The algorithm is implemented in the R-package simctest, available on CRAN.

Conservative hypothesis tests and confidence intervals using importance sampling

Importance sampling is a common technique for Monte Carlo approximation, including that of p-values. Here it is shown that a simple correction of the usual importance sampling p-values provides valid p-values, meaning that a hypothesis test created by rejecting the null hypothesis when the p-value is at most α will also have a Type I error rate of at most α. This correction uses the importance weight of the original observation, which gives valuable diagnostic information under the null hypothesis. Using the corrected p-values can be crucial for multiple testing and also in problems where evaluating the accuracy of importance sampling approximations is difficult. Inverting the corrected p-values provides a useful way to create Monte Carlo confidence intervals that maintain the nominal significance level and use only a single Monte Carlo sample.

Bootstrap confidence intervals and bias correction in the estimation of HIV incidence from surveillance data with testing for recent infection

The incidence of new infections is a key measure of the status of the HIV epidemic, but accurate measurement of incidence is often constrained by limited data. Karon et al. (Statist. Med. 2008; 27:4617–4633) developed a model to estimate the incidence of HIV infection from surveillance data with biologic testing for recent infection for newly diagnosed cases. This method has been implemented by public health departments across the United States and is behind the new national incidence estimates, which are about 40 per cent higher than previous estimates. We show that the delta method approximation given for the variance of the estimator is incomplete, leading to an inflated variance estimate. This contributes to the generation of overly conservative confidence intervals, potentially obscuring important differences between populations. We demonstrate via simulation that an innovative model-based bootstrap method using the specified model for the infection and surveillance process improves confidence interval coverage and adjusts for the bias in the point estimate. Confidence interval coverage is about 94–97 per cent after correction, compared with 96–99 per cent before. The simulated bias in the estimate of incidence ranges from −6.3 to +14.6 per cent under the original model but is consistently under 1 per cent after correction by the model-based bootstrap. In an application to data from King County, Washington in 2007 we observe correction of 7.2 per cent relative bias in the incidence estimate and a 66 per cent reduction in the width of the 95 per cent confidence interval using this method. We provide open-source software to implement the method that can also be extended for alternate models.

Secure key rate of the BB84 protocol using finite sample bits**A five page abstract of this paper appeared in: Sano Y, Matsumoto R and Uyematsu T 2010 Secure key rate of the BB84 protocol using finite sample bits ISIT 2010: Proc. IEEE Int. Symp. on Information Theory (13–18 June 2010, Austin, TX, USA) pp 2677–2681.

We improve the non-asymptotic key rate shown by Scarani and Renner by proposing several methods to construct tighter conservative confidence intervals of the phase error rate than one previously shown. In addition, we show that the accurate channel estimation method non-asymptotically increases the key rate over the amplitude damping channel as well as the asymptotic case in the BB84 protocol.

New Solution to Time Series Inference in Spurious Regression Problems

Phillips (1986) provides asymptotic theory for regressions that relate nonstationary time series including those integrated of order 1, I(1). A practical implication of related literature on spurious regression is that one cannot trust the usual confidence intervals. Therefore it is recommended that after carrying out unit root tests we work with differenced series instead of original data in levels. We propose a new alternative of using confidence intervals based on Maximum Entropy bootstrap explained in Vinod and Lopez-de-Lacalle (2009, J of Statistical Software). Extensive Monte Carlo simulations show that our proposal can provide more reliable conservative confidence intervals than traditional, differencing and block bootstrap (BB) intervals. We hope to let researchers avoid differencing the variables and work with their original specifications in levels.

Quantitative Genetic Dissection of Shoot Architecture Traits in Maize: Towards a Functional Genomics Approach

Quantitative trait loci (QTL) affecting the total number of leaves made before flowering and the number of leaves below the uppermost ear (NLBE) were mapped and characterized using the intermated B73 × Mo17 recombinant inbred lines (IBMRILs) of maize (Zea mays L.). B73 and Mo17 typically make 20 and 17 leaves, 14 and 11 of which are below the ear. Total number of leaves and the number of leaves below the uppermost ear are ∼80% heritable in the IBMRILs, which show strongly transgressive phenotypic ranges of 15 to 24 and 10 to 18 leaves for these traits. B73 alleles at loci in chromosome bins 1.06, 3.06, 4.08, 8.04, 8.05, 9.07, and 10.04 increase leaf numbers, with all but the 3.06 QTL affecting both of these highly correlated traits (r = 0.86, p &lt; 0.0001). Conservative QTL confidence intervals were computed and projected onto the draft maize genome sequence, revealing very narrow localizations (∼1 Mb) for four of the seven loci. More than 40% of the heritable variation for both traits is explained by an additive model, squarely accounting for the dramatic parental differences, but leaving the basis of the strong transgression unexplained. In addition, error rate control and confidence interval methods tailored for composite interval mapping are introduced, and their potential for improving QTL reporting is discussed.

On the conservative simultaneous confidence procedures for multiple comparisons among mean vectors

In this paper, conservative simultaneous confidence intervals for multiple comparisons among mean vectors in multivariate normal distributions are considered. Some properties of the multivariate Tukey–Kramer procedure for pairwise comparisons and the conservative simultaneous confidence procedure for comparisons with a control are presented. Particularly, the upper bound for the conservativeness of the simultaneous confidence procedure for comparisons with a control is obtained. Finally, numerical results by Monte Carlo simulations and an example to illustrate the procedure are given.

Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling

Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy. We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction. The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.

The Multivariate Tukey-Kramer Multiple Comparison Procedure Among Four Correlated Mean Vectors

SYNOPTIC ABSTRACTIn this article, conservative simultaneous confidence intervals for pairwise comparisons among mean vectors in multivariate normal distributions are considered. In order to give the simultaneous confidence intervals, we need the value of the upper 100α percentile of the T2max·p statistic. However, it is difficult to find the exact value. So, as an approximation procedure, Seo, Mano and Fujikoshi (1994) proposed the multivariate Tukey-Kramer procedure which is the multivariate version of Tukey-Kramer procedure (Tukey (1953); Kramer (1956, 1957)). Also, the multivariate version of the generalized Tukey conjecture has been affirmatively proved in the case of three correlated mean vectors by Seo, Mano and Fujikoshi (1994). In this article the affirmative proof of the multivariate generalized Tukey conjecture in the case of four mean vectors can be completed. Further, the upper bound for the conservativeness of the multivariate Tukey-Kramer procedure is also given. Finally, numerical results by Monte Carlo simulations and an example to illustrate the procedure are given.

Repeated confidence intervals for adaptive group sequential trials

This paper proposes a method for computing conservative confidence intervals for a group sequential test in which an adaptive design change is made one or more times over the course of the trial. The key idea, due to Müller and Schäfer (Biometrics 2001; 57:886-891), is that by preserving the null conditional rejection probability of the remainder of the trial at the time of each adaptive change, the overall type I error rate, taken unconditionally over all possible design modifications, is also preserved. We show how this principle may be extended to construct one-sided confidence intervals by applying the idea to a sequence of dual tests derived from the repeated confidence intervals (RCIs) proposed by Jennison and Turnbull (J. Roy. Statist. Soc. B 1989; 51:301-361). These adaptive RCIs, such as their classical counterparts, have the advantage that they preserve the desired coverage probability even if the pre-specified stopping rule is over-ruled. The statistical methodology is explored by simulations and is illustrated by an application to a clinical trial of deep brain stimulation for Parkinson's disease.

Turing's formula revisited*

A simple frequentist's justification of Turing's formula, an improvement to Turing's formula by means of reduced bias, a clarification of the relationships among various objects related to Turing's formula, a conservative confidence interval to Turing's target, and a conservative testing procedure using observed rank-frequencies under a hypothesized known infinite-dimensional multinomial distribution are given in this paper. As an example, the authorship of the nine-stanza poem “Shall I Die?” is tested against Shakespeare's canon and statistically significant evidence is found for a difference in word type usage.

베이지안 접근에 의한 모비율 선형함수의 신뢰구간

모비율에 대한 신뢰구간의 구축에 있어 정규근사에 의한 Wald 신뢰구간이 표준으로 인식되어 왔으나, 최근 여러 학자들에 의해 Wald 신뢰구간은 근사성에서 심각한 문제가 있다는 것이 밝혀지고 있어 Agresti와 Coull(1998)에 의해 제안된 방법이 새로운 표준이 되어 가고 있다. Agresti-Coull 방법은 간편하면서도 근사성 문제를 획기적으로 개선하였으나 모비율에 대한 여러 가지 문제에서 보수적인 신뢰구간을 제시하고 있다. 본 연구에서는 베이지안 접근 방법에 의해 Agresti-Coull 방법의 보수성을 개선한 모비율 선형 함수의 신뢰구간을 제시한다. It is known that Agresti-Coull approach is an effective tool for the construction of confidence intervals for various problems related to binomial proportions. However, the Agrest-Coull approach often produces a conservative confidence interval. In this note, confidence intervals based on a Bayesian approach are proposed for a linear function of independent binomial proportions. It is shown that the Bayesian confidence interval slightly outperforms the confidence interval based on Agresti-Coull approach in average sense.

Multiple comparisons of several homoscedastic multivariate populations

The limiting joint distribution of correlated Hotelling’s T 2 statistics associated with multiple comparisons with a control in multivariate one-way layout model is a multivariate central nonsingular chi-square distribution with one-factorial correlation matrix, which has the distribution function expressed in a closed form as an integral of a product of noncentral chi-square distribution functions with respect to a central chi-square density function. For pairwise comparisons, it is a multivariate central singular chi-square distribution whose distribution function is generally intricate. To overcome the complexity of the (exact or asymptotic) distribution theory of $$T^2_{\rm max}$$ -type statistics appeared in simultaneous confidence intervals of mean vectors, improved Bonferroni-type inequalities are applied to construct asymptotically conservative simultaneous confidence intervals for pairwise comparisons as well as comparisons with a control.