Coverage Probabilities Of Confidence Intervals Research Articles

Conditional bias adjusted estimator of treatment effect in 2-in-1 adaptive design

ABSTRACT For implementation of adaptive design, the adjustment of bias in treatment effect estimation becomes an increasingly important topic in recent years. While adaptive design literature traditionally focuses on the control of type I error rate and the adjustment of overall unconditional bias, the research on adjusting conditional bias has been limited. This paper proposes a conditional bias adjustment estimator of treatment effect under the context of 2-in-1 adaptive design and aims to provide a comprehensive investigation on their statistical properties including bias, mean squared error and coverage probability of confidence intervals. It demonstrated that conditional bias adjusted estimators greatly reduce the conditional bias and have similarly negligible unconditional bias compared with mean and median (unconditional) unbiased estimators. In addition, the test statistics is constructed based on the conditional bias adjustment estimators and compared with the naive unadjusted test.

Are state-space stock assessment model confidence intervals accurate? Case studies with SAM and Barents Sea stocks

Our main contribution is to examine the reliability of confidence intervals using the SAM state-space fish stock assessment model used for the assessment of many stocks by the International Council for the Exploration of the Seas. We focus on frequentist statistical inferences and more specifically on inference conditioned on specific values of the state-space model random effects drawn from their process distribution. This is somewhat consistent with simulation self-test procedures that are commonly used to examine the reliability of state-space assessment model results. However, recent research has indicated that some estimation bias may be expected in the conditional setting. Hence, we also investigate recently proposed bias corrected confidence intervals appropriate for the conditional inference setting. The SAM simulation coverage probabilities of 95% confidence intervals for SSB and Fbar were usually slightly larger than 95%, but in a small number of years these coverage probabilities could be much smaller than 95%. The bias corrected confidence intervals were more reliable. When averaged over years, the SAM and bias corrected confidence interval coverage probabilities were similar for the Northeast Artic cod and saithe case studies, but the bias corrected confidence intervals performed much better overall for the haddock case study.

Bootstrap-t Confidence Interval on Local Polynomial Regression Prediction

In local polynomial regression, prediction confidence interval estimation using standard theory will give coverage probability close to exact coverage probability. However, if the normality assumption is not met, the bootstrap method makes it possible to apply it. The working principle of the bootstrap method uses the resampling method where the sample data becomes a population and there is no need to know the distribution of the sample data is normal or not. Indiscriminate selection of smoothing parameters allows scatterplot results from local polynomial regressions to be rough and can even lead to misleading statistical conclusions. It is necessary to consider the optimal smoothing parameters to get local polynomial regression predictions that are not overfitting or underfitting. We offer two new algorithms based on the nested bootstrap resampling method to determine the bootstrap-t confidence interval in predicting local polynomial regression. Both algorithms consider the search for optimal smoothing parameters. The first algorithm performs paired and residual bootstrap samples, and the second algorithm performs based on residuals with residuals. The first algorithm provides a scatterplot and reasonable coverage probability on relatively large sample data. In contrast, the second algorithm is more powerful for each data size, including for relatively small sample data sizes. The mean of the bootstrap-t confidence interval coverage probability shows that the second algorithm for second-degree local polynomial regression is better than the other three. However, the larger the sample data size gives, the closer the closer the average coverage probability of the two algorithms is to the nominal coverage probability.

Modified-Lindley distribution and its applications to the real data

In this paper, a new three-parameter lifetime distribution is proposed by mixing modified Weibull and generalized gamma distributions. The point estimation on the distribution parameters are discussed through several estimators. The interval estimation is also studied with two methods based on asymptotic normality and likelihood ratio. A Monte Carlo simulation study is performed to evaluate the biases and mean square errors behaviors of point estimates for a different sample of size. A simulation study is also conducted to investigate the coverage probabilities of confidence intervals. The distribution modeling analyses are provided based on several real data sets to demonstrate the fitting ability of the introduced distribution.

Performance of statistical methods to address treatment non-adherence in pragmatic clinical trials with point-treatment settings: a simulation study

Introduction: The instrumental variable (IV)-based methods (e.g., two-stage least square [2SLS], two-stage residual inclusion [2SRI], and nonparametric causal bound [NPCB]) can be used to address non-adherence in pragmatic trials. These methods require assumptions, e.g., exclusion restriction, although they are known to handle unmeasured confounding. The inverse probability-weighted per-protocol [IPW-PP] method is useful in the same setting but requires different assumptions (no unmeasured confounding). Although all these methods aim to address the same problem, comprehensive simulations to compare their performance are absent in the literature. We performed extensive simulations when (1) confounding is present, (2) confounder is unmeasured but exclusion restriction is met, (3) exclusion restriction is violated, and (4) non-adherence is one-sided and differential.
 Method: We compared the performance in terms of bias, standard error (SE), mean squared error (MSE), and 95% confidence interval coverage probability.
 Results: For setting-1, IPW-PP outperforms IV-methods in terms of bias, SE, MSE, and coverage for <80% non-adherence but produces high bias beyond that point. IPW-PP also has high biases, but 2SLS and 2SRI work well for setting-2. For setting-3, 2SLS and 2SRI perform the worst in all scenarios; IPW-PP produces unbiased estimates when necessary confounders are measured and adjusted. For setting-4, IPW-PP has less bias, but 2SLS and 2SRI have higher SE and MSE. NPCB has wider bounds in all scenarios. We also analyze a two-arm trial to estimate the effect of vitamin A supplementation on childhood mortality after addressing non-adherence.
 Conclusion: We need to be cautious using the IPW-PP when non-adherence is very high or strong unmeasured confounding and should avoid using the IV methods when the exclusion restriction assumption is violated or high differential non-adherence. Since assumptions are different and often untestable for IPW-PP and IV methods, we suggest analyzing data using both methods for a robust conclusion.

Generalized Inference on Stress-Strength Reliability in Generalized Pareto Model

The article focuses on the inference of stress-strength reliability in generalized Pareto model using the generalized variable approach and bootstrap percentile method. Simulation studies are conducted to obtain expected lengths and coverage probabilities of confidence intervals constructed using the generalized variable and the bootstrap percentile methods. An example consisting of real stress-strength data is also presented for illustrative purposes.

Practical Review and Comparison of Modified Covariance Estimators for Linear Mixed Models in Small‐sample Longitudinal Studies with Missing Data

SummaryMixed‐effects models for repeated measures (MMRMs) with an ‘unstructured’ (UN) covariance structure are frequently used in primary analyses for group comparisons of incomplete continuous longitudinal data from drug development trials. However, MMRM‐UN analysis could lead to convergence problems in numerical optimisation, especially in trials with a small sample size or high dropout rate. Although the so‐called sandwich covariance estimator is robust against the misspecification of the covariance structure, its performance deteriorates for small sample sizes. We review eight modified covariance estimators adjusted for small‐sample bias and compare their performances in the framework of MMRM analysis through simulations. In terms of the type 1 error rate and coverage probability of confidence intervals for group comparisons, Mancl and DeRouen's covariance estimator (MD) shows the best performance among the modified covariance estimators, followed by Fay and Graubard's estimator. The performance of MD is nearly equivalent to that of the Kenward–Roger method with a UN structure. The Kenward–Roger method with first‐order autoregressive structure results in substantial inflation of the type 1 error rate in the scenario where the variance of measurements increases across visits. In summary, we recommend the use of MD in MMRM analysis if the convergence problem involving a UN structure occurs in small clinical trials.

Correction to ‘Distribution‐free Approximate Methods for Constructing Confidence Intervals for Quantiles’

One way of evaluating the quality of a simulation study methodology is to estimate the Monte Carlo Error (MCE). The formula for this calculation was incorrectly implemented in Nagaraja & Nagaraja (2020). The correct formula that replaces equation (24) on page 96 of the original paper is as follows. The MCE was estimated for the confidence interval coverage probability calculations, and those results have been updated in Tables 29–56 of the Supporting Information. No conclusions changed due to this correction. Supporting info item Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Asymptotic distributions and performance of empirical skewness measures

A number of skewness measures have been proposed and applied to theoretical distributions. However, the corresponding empirical counterparts have been analyzed only rarely, especially with respect to their asymptotic properties and limit distributions. Six of these empirical measures are considered. After discussing some general properties, the limiting distribution for each measure is derived under weak assumptions. The performance of these estimators is analyzed in simulations using tests and the coverage probabilities of confidence intervals. A particular focus is put on the standardized central third moment as the most popular measure of skewness. Since it turns out to behave poorly, especially when sample sizes are small, the use of alternative and more suitable skewness measures is recommended. A real data application illustrates some of the findings.

The influence of nonnormality from primary studies on the standardized mean difference in meta-analysis.

In this study we investigated the influence of data nonnormality in the primary studies on meta-analysis of the standardized mean difference (SMD) for a two-independent-group design. The bias, mean squared error, and confidence interval coverage probability of the mean effect sizes under different types of population distributions were compared. Also, the performance of the Q test was examined. The results showed that oppositely skewed distributions (i.e., distributions skewed in different directions) showed poor performance for point and interval estimates of mean effect sizes in meta-analysis, especially when the tails were pointing toward each other. The previously found adverse impacts due to nonnormality in primary studies do not disappear when primary studies with nonnormal data are meta-analyzed, even when the average sample size and number of studies are large. The results also showed that, when the tails were pointing toward each other, the Type I error rates of the Q test were inflated. We suggest that the impact of violating the assumption of normality should not be ignored in meta-analysis.

Computational issues in fitting joint frailty models for recurrent events with an associated terminal event

Background and objective: Joint frailty regression models are intended for the analysis of recurrent event times in the presence of informative drop-outs. They have been proposed for clinical trials to estimate the effect of some treatment on the rate of recurrent heart failure hospitalisations in the presence of drop-outs due to cardiovascular death. Whereas a R-software-package for fitting joint frailty models is available, some technical issues have to be solved in order to use SASⓇ11SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. software, which is required in the regulatory environment of clinical trials.Methods: First, we demonstrate how to solve these issues by deriving proper likelihood-decompositions, in particular for the case of non-normally distributed random terms. Second, we perform a simulation study to evaluate the accuracy of different software-implementations (in SAS and R) in terms of convergence behavior, bias of model parameter estimates and coverage probabilities of confidence intervals. Therefore we developed SAS macros that facilitate the analysis and simulation of joint frailty data. These are provided as supplementary material along with comprehensive manuals.Results: Whereas estimates for regression coefficients are unbiased irrespective of the software, the bias of the remaining (nuisance) parameter estimates strongly depends on the software: SAS is shown to be much more efficient in avoiding bias compared to R. However, even in SAS a careful choice of the implementation is required to get reliable results, in particular for the joint gamma frailty model. By far the best performance is reached with a SAS-implementation that makes use of the probability integral transformation method.Conclusions: We have shown, that getting reliable results from joint frailty models is not straightforward and users should be aware about the computational options between and within software packages. Based on our simulation study, we elaborate recommendations on these options. In addition, our provided SAS macros may encourage statistical practitioners to apply these models in clinical trials with recurrent event data and potentially informative drop-outs.

Estimating population size with imperfect detection using a parametric bootstrap

We develop a novel method of estimating population size from imperfectly detected counts of individuals and a separate estimate of detection probability. Observed counts are separated into classes within which detection probability is assumed constant. Within a detection class, counts are modeled as a single binomial observation X with success probability p where the goal is to estimate index N. We use a Horvitz–Thompson‐like estimator for N and account for uncertainty in both sample data and estimated success probability via a parametric bootstrap. Unlike capture–recapture methods, our model does not require repeated sampling of the population. Our method is able to achieve good results, even with small X. We show in a factorial simulation study that the median of the bootstrapped sample has small bias relative to N and that coverage probabilities of confidence intervals for N are near nominal under a wide array of scenarios. Our methodology begins to break down when P(X=0)>0.1 but is still capable of obtaining reasonable confidence coverage. We illustrate the proposed technique by estimating (1) the size of a moose population in Alaska and (2) the number of bat fatalities at a wind power facility, both from samples with imperfect detection probabilities, estimated independently.

Confidence, prediction, and tolerance in linear mixed models.

The literature about Prediction Interval (PI) and Tolerance Interval (TI) in linear mixed models is usually developed for specific designs, which is a main limitation to their use. This paper proposes to reformulate the two‐sided PI to be generalizable under a wide variety of designs (one random factor, nested and crossed designs for multiple random factors, and balanced or unbalanced designs). This new methodology is based on the Hessian matrix, namely, the inverse of (observed) Fisher Information matrix, and is built with a cell mean model. The degrees of freedom for the total variance are calculated with the generalized Satterthwaite method and compared to the Kenward‐Roger's degrees of freedom for fixed effects. Construction of two‐sided TIs are also detailed with one random factor, and two nested and two crossed random variables. An extensive simulation study is carried out to compare the widths and coverage probabilities of Confidence Intervals (CI), PIs, and TIs to their nominal levels. It shows excellent coverage whatever the design and the sample size are. Finally, these CIs, PIs, and TIs are applied to two real data sets: one from orthopedic surgery study (intralesional resection risk) and the other from assay validation study during vaccine development.

Nonparametric estimation of the cumulative incidence function under outcome misclassification using external validation data.

Misclassification of outcomes or event types is common in health sciences research and can lead to serious bias when estimating the cumulative incidence functions in settings with competing risks. Recent work has shown how to estimate nonparametric cumulative incidence functions in the presence of nondifferential outcome misclassification when the misclassification probabilities are known. Here, we extend this approach to account for misclassification that is differential with respect to important predictors of the outcome using misclassification probabilities estimated from external validation data. Moreover, we propose a bootstrap approach in which the observations from both the main study data and the external validation study are resampled to allow the uncertainty in the misclassification probabilities to propagate through the analysis into the final confidence intervals, ensuring appropriate confidence interval coverage probabilities. The proposed estimator is shown to be uniformly consistent and simulation studies indicate that both the estimator and the standard error estimation approach perform well in finite samples. The methodology is applied to estimate the cumulative incidence of death and disengagement from HIV care in a large cohort of HIV infected individuals in sub-Saharan Africa, where a significant death underreporting issue leads to outcome misclassification. This analysis uses external validation data from a separate study conducted in the same country.

On mean-based bivariate Birnbaum-Saunders distributions: Properties, inference and application

Birnbaum-Saunders models have been widely used to describe data following positive-skew distributions. In this article, we introduce a bivariate Birnbaum-Saunders distribution which has the mean as one of its parameters, allowing us to mimic the standard parameterization of the normal distribution, but in an asymmetric framework. We derive some properties of the mean-based bivariate Birnbaum-Saunders distribution useful for statistical and reliability analyses. Maximum likelihood and modified moment estimations of the model parameters and associated inference are considered. A simulation study is conducted to evaluate the performance of the corresponding estimators and coverage probabilities of confidence intervals are also discussed. Finally, a real-world data analysis is carried out for illustrating the potential of the proposed model.

A unified method for improved inference in random effects meta-analysis.

Random effects meta-analyses have been widely applied in evidence synthesis for various types of medical studies. However, standard inference methods (e.g. restricted maximum likelihood estimation) usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations; for instance, coverage probabilities of confidence intervals can be substantially below the nominal level. The main reason is that these inference methods rely on large sample approximations even though the number of synthesized studies is usually small or moderate in practice. In this article, we solve this problem using a unified inference method based on Monte Carlo conditioning for broad application to random effects meta-analysis. The developed method provides improved confidence intervals with coverage probabilities that are closer to the nominal level than standard methods. As specific applications, we provide new inference procedures for three types of meta-analysis: conventional univariate meta-analysis for pairwise treatment comparisons, meta-analysis of diagnostic test accuracy, and multiple treatment comparisons via network meta-analysis. We also illustrate the practical effectiveness of these methods via real data applications and simulation studies.

The large sample coverage probability of confidence intervals in general regression models after a preliminary hypothesis test

AbstractWe derive a computationally convenient formula for the large sample coverage probability of a confidence interval for a scalar parameter of interest following a preliminary hypothesis test that a specified vector parameter takes a given value in a general regression model. Previously, this large sample coverage probability could only be estimated by simulation. Our formula only requires the evaluation, by numerical integration, of either a double or a triple integral, irrespective of the dimension of this specified vector parameter. We illustrate the application of this formula to a confidence interval for the odds ratio of myocardial infarction when the exposure is recent oral contraceptive use, following a preliminary test where two specified interactions in a logistic regression model are zero. For this real‐life data, we compare this large sample coverage probability with the actual coverage probability of this confidence interval, obtained by simulation.

Model averaging for M-estimation

ABSTRACTM-estimation is a widely used technique for robust statistical inference. In this paper, we study model selection and model averaging for M-estimation to simultaneously improve the coverage probability of confidence intervals of the parameters of interest and reduce the impact of heavy-tailed errors or outliers in the response. Under general conditions, we develop robust versions of the focused information criterion and a frequentist model average estimator for M-estimation, and we examine their theoretical properties. In addition, we carry out extensive simulation studies as well as two real examples to assess the performance of our new procedure, and find that the proposed method produces satisfactory results.

Bootstrap Methods in Econometrics

The bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one's data or a model estimated from the data. Under conditions that hold in a wide variety of econometric applications, the bootstrap provides approximations to distributions of statistics, coverage probabilities of confidence intervals, and rejection probabilities of hypothesis tests that are more accurate than the approximations of first-order asymptotic distribution theory. The reductions in the differences between true and nominal coverage or rejection probabilities can be very large. In addition, the bootstrap provides a way to carry out inference in certain settings where obtaining analytic distributional approximations is difficult or impossible. This article explains the usefulness and limitations of the bootstrap in contexts of interest in econometrics. The presentation is informal and expository. It provides an intuitive understanding of how the bootstrap works. Mathematical details are available in the references that are cited.

Coverage probabilities of confidence intervals for the slope parameter of linear regression model when the error term is not normally distributed

Confidence interval for the slope parameter based on the t-statistic is not appropriate when the error term is not normally distributed. In this paper, we examine some existing methods for the interval estimation of the slope parameter and suggest new methods for giving better coverage accuracy. The following methods are considered: the standard t, the bootstrap-t, and three intervals based on the transformations of the t-statistic. The results for the Exponential and Weibull distributions and for the real data set are presented. The simulation study shows that these introduced intervals give better results than existing methods.