Confidence Interval Procedures Research Articles

Single criterion for rating confidence interval procedures: a generalization to stochastic stopping rules

In this paper, we propose a generalization of the VAMP1RE criterion, which was used to rank and rate fixed sample size confidence interval procedures (CIPs), to be able to rank CIPs that use stochastic stopping rules. The key aspects of VAMP1RE remain unchanged in the generalized version. Specifically, we intend for it to serve as a single criterion, which measures the mean squared error of the difference between the coverage values of the CIP of interest and an ideal CIP. Moreover, we advocate that the quality of an interval should be evaluated based on its similarity to the interval produced by the ideal CIP, rather than solely on its width. We estimate the criterion using Monte Carlo simulation with Schruben’s coverage function. The generalization involves two modifications. The first modification replaces the arbitrary selection of nominal coverage values with averaging over a specified distribution of such values. The second modification replaces the use of a fixed sample size with a stopping condition that depends on the CIP of interest. We present experimental results that show how the generalized VAMP1RE criterion can be used to rate CIPs with stochastic stopping rules by providing useful numerical values.

Overlapping Batch Confidence Intervals on Statistical Functionals Constructed from Time Series: Application to Quantiles, Optimization, and Estimation

We propose a general purpose confidence interval procedure (CIP) for statistical functionals constructed using data from a stationary time series. The procedures we propose are based on derived distribution-free analogues of the χ 2 and Student’s t random variables for the statistical functional context, and hence apply in a wide variety of settings including quantile estimation, gradient estimation, M-estimation, CVAR-estimation, and arrival process rate estimation, apart from more traditional statistical settings. Like the method of subsampling, we use overlapping batches of time series data to estimate the underlying variance parameter; unlike subsampling and the bootstrap, however, we assume that the implied point estimator of the statistical functional obeys a central limit theorem (CLT) to help identify the weak asymptotics (called OB-x limits, x=I,II,III) of batched Studentized statistics. The OB-x limits, certain functionals of the Wiener process parameterized by the size of the batches and the extent of their overlap, form the essential machinery for characterizing dependence, and consequently the correctness of the proposed CIPs. The message from extensive numerical experimentation is that in settings where a functional CLT on the point estimator is in effect, using large overlapping batches alongside OB-x critical values yields confidence intervals that are often of significantly higher quality than those obtained from more generic methods like subsampling or the bootstrap. We illustrate using examples from CVaR estimation, ARMA parameter estimation, and NHPP rate estimation; R and MATLAB code for OB-x critical values is available at web.ics.purdue.edu/ ∼ pasupath .

Inference for partial correlations of a multivariate Gaussian time series

Abstract We derive an asymptotic joint distribution and novel covariance estimator for the partial correlations of a multivariate Gaussian time series under mild regularity conditions. Using our derived asymptotic distribution, we develop a Wald confidence interval and testing procedure for inference of individual partial correlations for time series data. Through simulation we demonstrate that our proposed confidence interval attains higher coverage rates and our testing procedure achieves false positive rates closer to the nominal levels than approaches that assume independent observations when autocorrelation is present.

Impact of Bias Correction of the Least Squares Estimation on Bootstrap Confidence Intervals for Bifurcating Autoregressive Models

The least squares (LS) estimator of the autoregressive coefficient in the bifurcating autoregressive (BAR) model was recently shown to suffer from substantial bias, especially for small to moderate samples. This study investigates the impact of the bias in the LS estimator on the behavior of various types of bootstrap confidence intervals for the autoregressive coefficient and introduces methods for constructing bias-corrected bootstrap confidence intervals. We first describe several bootstrap confidence interval procedures for the autoregressive coefficient of the BAR model and present their bias-corrected versions. The behavior of uncorrected and corrected confidence interval procedures is studied empirically through extensive Monte Carlo simulations and two real cell lineage data applications. The empirical results show that the bias in the LS estimator can have a significant negative impact on the behavior of bootstrap confidence intervals and that bias correction can significantly improve the performance of bootstrap confidence intervals in terms of coverage, width, and symmetry.

Identification of Maximum Safe Dose Based on Ratio of Normally Distributed Data under Heteroscedasticity

In most of the various stepwise confidence interval procedures formulated for identifying maximum safe dose (MSD), homogeneity of variances among different dose levels were required. But in practice, homogeneity of variance is often in doubt. This paper proposes a stepwise confidence set procedure for identifying MSD of drugs based on ratio of population means for normally distributed data under heteroscedasticity without the need for multiplicity adjustment. The procedure employed Fieller's method and obtained individual (1-α)100% confidence intervals for identification of the MSD. We illustrate the procedure with a real life example. In addition, we show that power of the procedure increases with increasing ratio of means, and sample size. Power however decreases with increase in clinical relevance margins. We also illustrate that the new procedure can properly control familywise error rate (FWER).

A nonparametric approach to confidence intervals for concordance index and difference between correlated indices

ABSTRACT Concordance refers to the probability that subjects with high values on one variable also have high values on another variable. This index has wide application in practice, as a measure of effect size in group-comparison studies, an index of accuracy in diagnostic studies, and a discrimination index for prediction models. Herein, we provide a unified framework for statistical inference involving concordance indices for standard variables of binary, ordinal, and continuous types. In particular, we develop confidence interval procedures for a single concordance index and differences between two correlated indices. Simulation results show that procedures based on logit-transformation for a single index and Fisher’s -transformation for a difference between indices perform very well in terms of coverage and tail errors even when the sample size is as small as 30, unless the concordance is high and the standard is a binary variable for which at least 50 subjects are needed. We illustrate the procedures for a variety of standard variables with previously published data. Illustrative SAS code is provided.

Confidence Intervals After Variance Stabilization May Go Head-to-Head with Exact Confidence Intervals: A New Class of Illustrations

We begin with an overview on variance stabilizing transformations (VST) along with three classical examples for completeness: the arcsine, square-root and Fisher's z-transformations (Examples 1–3). Then, we construct three new examples (Examples 4–6) of VST-based and central limit theorem (CLT)’based large-sample confidence interval methodologies. These are special examples in the sense that in each situation, we also have an exact confidence interval procedure for the parameter of interest. Tables 1–3 obtained exclusively under Examples 4–6 via exact calculations show that the VST-based (a) large-sample confidence interval methodology wins over the CLT-based large-sample confidence interval methodology, (b) confidence intervals’ exact coverage probabilities are better than or nearly same as those associated with the exact confidence intervals and (c) intervals are never wider (in the log-scale) than the CLT-based intervals across the board. A possibility of such a surprising behaviour of the VST-based confidence intervals over the exact intervals was not on our radar when we began this investigation. Indeed the VST-based inference methodologies may do extremely well, much more so than the existing literature reveals as evidenced by the new Examples 4–6. AMS subject classifications: 62E20; 62F25; 62F12

Confidence as Likelihood

Confidence and likelihood are fundamental statistical concepts with distinct technical interpretation and usage. Confidence is a meaningful concept of uncertainty within the context of confidence-interval procedure, while likelihood has been used predominantly as a tool for statistical modelling and inference given observed data. Here we show that confidence is in fact an extended likelihood, thus giving a much closer correspondence between the two concepts. This result gives the confidence concept an external meaning outside the confidence-interval context, and vice versa, it gives the confidence interpretation to the likelihood. In addition to the obvious interpretation purposes, this connection suggests two-way transfers of technical information. For example, the extended likelihood theory gives a clear way to update or combine confidence information. On the other hand, the confidence connection means that intervals derived from the extended likelihood have the same status as confidence intervals. This gives the extended likelihood direct access to the frequentist probability, an objective certification not directly available to the classical likelihood.

Improved procedures and computer programs for equivalence assessment of correlation coefficients.

The correlation coefficient is the most commonly used measure for summarizing the magnitude and direction of linear relationship between two response variables. Considerable literature has been devoted to the inference procedures for significance tests and confidence intervals of correlations. However, the essential problem of evaluating correlation equivalence has not been adequately examined. For the purpose of expanding the usefulness of correlational techniques, this article focuses on the Pearson product-moment correlation coefficient and the Fisher's z transformation for developing equivalence procedures of correlation coefficients. Equivalence tests are proposed to assess whether a correlation coefficient is within a designated reference range for declaring equivalence decisions. The important aspects of Type I error rate, power calculation, and sample size determination are also considered. Special emphasis is given to clarify the nature and deficiency of the two one-sided tests for detecting a lack of association. The findings demonstrate the inappropriateness of existing methods for equivalence appraisal and validate the suggested techniques as reliable and primary tools in correlation analysis.

Confidence interval estimation for treatment effects in cluster randomization trials based on ranks.

A cluster randomization trial is one in which clusters of individuals are randomly allocated to different intervention arms. This design has become the standard for the evaluation of health care and educational strategies. To assess treatment effect, many cluster randomization trials involve outcomes that are lack meaningful units, making interpretation difficult. This difficulty may be dealt with by estimating the Mann-Whitney probability, which quantifies the probability that a typical response from one treatment arm is larger (or smaller) than a typical response from the other arm. In this work, we propose procedures for estimating this probability in cluster randomization trials. Primary emphasis is given to confidence interval estimation in trials with a small number of large clusters. The essence of the procedures is to obtain placement values based on overall ranks and arm-specific ranks prior to application of the ratio estimator, cluster-size-weighted means and mixed models for adjusting clustering effects. Nine confidence intervals were developed by applying three interval methods each based on the three variance estimators. The proposed methods can be applied to studies with binary, ordinal or continuous outcomes without making parametric assumptions. Simulation results demonstrated that the three variance estimators performed equally well, with the confidence interval procedures based on logit and inverse hyperbolic sine transformations performing better in terms of coverage and average interval width, even when the numbers of clusters are as small as 3 to 5 clusters per arm. The methods are illustrated using data from three published cluster randomization trials with SAS code provided.

Testing and dating structural changes in copula-based dependence measures

This paper is concerned with testing and dating structural breaks in the dependence structure of multivariate time series. We consider a cumulative sum (CUSUM) type test for constant copula-based dependence measures, such as Spearman's rank correlation and quantile dependencies. The asymptotic null distribution is not known in closed form and critical values are estimated by an i.i.d. bootstrap procedure. We analyze size and power properties in a simulation study under different dependence measure settings, such as skewed and fat-tailed distributions. To date breakpoints and to decide whether two estimated break locations belong to the same break event, we propose a pivot confidence interval procedure. Finally, we apply the test to the historical data of 10 large financial firms during the last financial crisis from 2002 to mid-2013.

Building Discriminate Function-Review

Discriminant Analysis has been widely used to classify data into subgroups based on certain criteria. The classification process depends on choosing any variable that shows a statistical significance, then use the selected variables to build the discriminant function. In order to investigate the statistical significance of the variables in our data, we used Roy-Bose procedure for finding confidence intervals and t-test, which is one of the popular variable-selection methods in discriminant analysis. In addition, some other variable-selection techniques has been employed, namely, Forward-Selection, Backward-Selection, and Stepwise-Selection methods, which are usually used to select variables in linear regression analysis. Furthermore, a principal component analysis has been carried out for the purpose of choosing the variables with high statistical significance. The selected variables have been used to build the discriminant function.

Adaptive filtering and analysis of EEG signals in the time-frequency domain based on the local entropy

The brain dynamics in the electroencephalogram (EEG) data are often challenging to interpret, specially when the signal is a combination of desired brain dynamics and noise. Thus, in an EEG signal, anything other than the desired electrical activity, which is produced due to coordinated electrochemical process, can be considered as unwanted or noise. To make brain dynamics more analyzable, it is necessary to remove noise in the temporal location of interest, as well as to denoise data from a specific spatial location. In this paper, we propose a novel method for noisy EEG analysis with accompanying toolbox which includes adaptive, data-driven noise removal technique based on the improved intersection of confidence interval (ICI)-based algorithm. Next, a local entropy-based method for EEG data analysis was designed and included in the toolbox. As shown in the paper, the relative intersection of confidence interval (RICI) procedure retains the dominant dipole activity projected on electrodes, while the local (short-term) Rényi entropy-based analysis of the EEG representation in the time-frequency domain is efficient in detecting the presence of P300 event-related potential (ERP) at specific electrodes. Namely, the P300 are detected as sharp drop of entropy in the temporal domain that enabled accurate calculation of the index of the noise class for the EEG signals.

Accurate reliability inference based on Wiener process with random effects for degradation data

The Wiener process is often used to fit degradation data in reliability modeling. Though there is an abundant literature covering the inference procedures for the Wiener model, their performance may not be satisfactory when the sample size is small, which is often the case in degradation data analysis. In this paper, we focus on the accurate reliability inference based on the Wiener process with random drift parameter for degradation data. We propose an exact procedure to test whether there is population heterogeneity. An exact confidence interval (CI) procedure for the diffusion parameter of the Wiener process is also obtained. Generalized confidence intervals (GCIs) are proposed for model parameters and some commonly used reliability metrics such as the quantile, the reliability function, etc. Furthermore, a generalized prediction interval (GPI) for the future degradation levels is obtained. The performance of the proposed GCIs and GPI is assessed by the Monte Carlo simulation. The simulation results show that the proposed interval procedures outperform existing method such as the Wald, the bootstrap-p and the likelihood-ratio-based CIs in terms of the coverage probability, and our proposed procedures have desirable properties even under small sample sizes. Finally, an example is used to illustrate the proposed procedures.

Constructing a confidence interval for the fraction who benefit from treatment, using randomized trial data.

The fraction who benefit from treatment is the proportion of patients whose potential outcome under treatment is better than that under control. Inference on this parameter is challenging since it is only partially identifiable, even in our context of a randomized trial. We propose a new method for constructing a confidence interval for the fraction, when the outcome is ordinal or binary. Our confidence interval procedure is pointwise consistent. It does not require any assumptions about the joint distribution of the potential outcomes, although it has the flexibility to incorporate various user-defined assumptions. Our method is based on a stochastic optimization technique involving a second-order, asymptotic approximation that, to the best of our knowledge, has not been applied to biomedical studies. This approximation leads to statistics that are solutions to quadratic programs, which can be computed efficiently using optimization tools. In simulation, our method attains the nominal coverage probability or higher, and can have narrower average width than competitor methods. We apply it to a trial of a new intervention for stroke.

Exact Adaptive Confidence Intervals for Small Areas

Abstract In the analysis of survey data, it is of interest to estimate and quantify uncertainty about means or totals for each of several nonoverlapping subpopulations or areas. When the sample size for a given area is small, standard confidence intervals based on data only from that area can be unacceptably wide. In order to reduce interval width, practitioners often utilize multilevel models in order to borrow information across areas, resulting in intervals centered around shrinkage estimators. However, such intervals only have the nominal coverage rate on average across areas under the assumed model for across-area heterogeneity. The coverage rate for a given area depends on the actual value of the area mean and can be nearly zero for areas with means that are far from the across-group average. As such, the use of uncertainty intervals centered around shrinkage estimators are inappropriate when area-specific coverage rates are desired. In this article, we propose an alternative confidence interval procedure for area means and totals under normally distributed sampling errors. This procedure not only has constant 1−α frequentist coverage for all values of the target quantity but also uses auxiliary information to borrow information across areas. Because of this, the corresponding intervals have shorter expected lengths than standard confidence intervals centered on the unbiased direct estimator. Importantly, the coverage of the procedure does not depend on the assumed model for across-area heterogeneity. Rather, improvements to the model for across-area heterogeneity result in reduced expected interval width.

A METHOD FOR ESTIMATING THE PROPORTION OF HIV INFECTED PERSONS THAT HAVE BEEN DIAGNOSED AND APPLICATION TO CHINA.

Estimation of the proportion of living HIV infected persons that have been diagnosed is critical for tracking progress toward meeting the UNAIDS goal that all persons who need HIV treatment receive it. The objective of this article is to develop a method for estimating that proportion. The methodological problem is that persons with undiagnosed HIV infection are not directly observable and are a "hidden" population. Here we propose a methodology for estimating the proportion diagnosed that is relatively simple to implement. The key idea is that in many settings certain health conditions such as pregnancy or an upcoming surgery lead to mandatory HIV tests. The size of the undiagnosed infected population can be estimated from the numbers of infected persons diagnosed by mandatory tests and an estimate of the rate that persons in the undiagnosed infected population receive mandatory tests. We discuss approaches for estimating the rate of mandatory testing in the undiagnosed population, such as surgical or pregnancy rates. We develop estimators of the proportion diagnosed and confidence interval procedures. Sample size considerations and sensitivity analyses to underlying assumptions are considered. The proposed methods can be performed at a local level and within demographic strata. Implementation of the method is simple and requires neither historical HIV/AIDS surveillance data nor biomarkers such as CD4 cell counts. The methods are applied to data from Dehong Prefecture in Yunnan Province, China.

Exact adaptive confidence intervals for linear regression coefficients

We propose an adaptive confidence interval procedure (CIP) for the coefficients in the normal linear regression model. This procedure has a frequentist coverage rate that is constant as a function of the model parameters, yet provides smaller intervals than the usual interval procedure, on average across regression coefficients. The proposed procedure is obtained by defining a class of CIPs that all have exact $1-\alpha $ frequentist coverage, and then selecting from this class the procedure that minimizes a prior expected interval width. We describe an adaptive approach for estimating the prior distribution from the data, so that the potential risk of a poorly specified prior is reduced. The resulting adaptive confidence intervals maintain exact non-asymptotic $1-\alpha $ coverage if two conditions are met - that the design matrix is full rank (which will be known) and that the errors are normally distributed (which can be checked empirically). No assumptions on the unknown parameters are necessary to maintain exact coverage. Additionally, in a “$p$ growing with $n$” asymptotic scenario, this adaptive FAB procedure is asymptotically Bayes-optimal among $1-\alpha $ frequentist CIPs.

Inference on treatment effect modification by biomarker response in a three-phase sampling design.

An objective in randomized clinical trials is the evaluation of "principal surrogates," which consists of analyzing how the treatment effect on a clinical endpoint varies over principal strata subgroups defined by an intermediate response outcome under both or one of the treatment assignments. The latter effect modification estimand has been termed the marginal causal effect predictiveness (mCEP) curve. This objective was addressed in two randomized placebo-controlled Phase 3 dengue vaccine trials for an antibody response biomarker whose sampling design rendered previously developed inferential methods highly inefficient due to a three-phase sampling design. In this design, the biomarker was measured in a case-cohort sample and a key baseline auxiliary strongly associated with the biomarker (the "baseline surrogate measure") was only measured in a further sub-sample. We propose a novel approach to estimation of the mCEP curve in such three-phase sampling designs that avoids the restrictive "placebo structural risk" modeling assumption common to past methods and that further improves robustness by the use of non-parametric kernel smoothing for biomarker density estimation. Additionally, we develop bootstrap-based procedures for pointwise and simultaneous confidence intervals and testing of four relevant hypotheses about the mCEP curve. We investigate the finite-sample properties of the proposed methods and compare them to those of an alternative method making the placebo structural risk assumption. Finally, we apply the novel and alternative procedures to the two dengue vaccine trial data sets.

Confidence intervals of the Mann-Whitney parameter that are compatible with the Wilcoxon-Mann-Whitney test.

For the two-sample problem, the Wilcoxon-Mann-Whitney (WMW) test is used frequently: it is simple to explain (a permutation test on the difference in mean ranks), it handles continuous or ordinal responses, it can be implemented for large or small samples, it is robust to outliers, it requires few assumptions, and it is efficient in many cases. Unfortunately, the WMW test is rarely presented with an effect estimate and confidence interval. A natural effect parameter associated with this test is the Mann-Whitney parameter, φ = Pr[ X<Y ] + 0.5 Pr[X = Y ]. Ideally, we desire confidence intervals on φ that are compatible with the WMW test, meaning the test rejects at level α if and only if the 100(1 - α)% confidence interval on the Mann-Whitney parameter excludes 1/2. Existing confidence interval procedures on φ are not compatible with the usual asymptotic implementation of the WMW test that uses a continuity correction nor are they compatible with exact WMW tests. We develop compatible confidence interval procedures for the asymptotic WMW tests and confidence interval procedures for some exact WMW tests that appear to be compatible. We discuss assumptions and interpretation of the resulting tests and confidence intervals. We provide the wmwTest function of the asht R package to calculate all of the developed confidence intervals.