Coverage Probabilities Of Confidence Intervals Research Articles

On Bayesian estimation of a latent trait model defined by a rank-based likelihood

Maximum likelihood estimation (frequentist) and Bayesian estimation are two common parameter estimation methods. However, maximum likelihood estimation faces limitations, including the effect of outliers, computational complexity, and issues with ordinal categorical data, leading to biased estimates and inaccurate coverage probabilities. To address these limitations, this study employed a latent trait model with a Bayesian marginal likelihood of rank-based estimation for parameter estimation. The simulation results demonstrated favorable performance of the proposed method. Trace plots of all parameters showed good distribution and quick convergence, with the potential scale reduction factor not exceeding 1, indicating no convergence issues. Furthermore, the posterior predictive check showed the simulated data closely resembled the observed data, indicating the method effectively captures within-region variation through a latent trait parameter. Performance metrics like mean absolute error, root mean square error, and 95% confidence interval coverage probability revealed the estimates from the proposed Bayesian method surpassed those from classical approaches. In conclusion, a latent traits model with Bayesian marginal likelihood and rank-based estimation is considered a superior parameter estimation technique compared to classical methods, particularly for dealing with ordinal categorical data.

Pairwise Accelerated Failure Time Regression Models for Infectious Disease Transmission in Close-Contact Groups With External Sources of Infection.

Many important questions in infectious disease epidemiology involve associations between covariates (e.g., age or vaccination status) and infectiousness or susceptibility. Because disease transmission produces dependent outcomes, these questions are difficult or impossible to address using standard regression models from biostatistics. Pairwise survival analysis handles dependent outcomes by calculating likelihoods in terms of contact interval distributions in ordered pairs of individuals. The contact interval in the ordered pair is the time from the onset of infectiousness in to infectious contact from to , where an infectious contact is sufficient to infect if they are susceptible. Here, we introduce a pairwise accelerated failure time regression model for infectious disease transmission that allows the rate parameter of the contact interval distribution to depend on individual-level infectiousness covariates for , individual-level susceptibility covariates for , and pair-level covariates (e.g., type of relationship). This model can simultaneously handle internal infections (caused by transmission between individuals under observation) and external infections (caused by environmental or community sources of infection). We show that this model produces consistent and asymptotically normal parameter estimates. In a simulation study, we evaluate bias and confidence interval coverage probabilities, explore the role of epidemiologic study design, and investigate the effects of model misspecification. We use this regression model to analyze household data from Los Angeles County during the 2009 influenza A (H1N1) pandemic, where we find that the ability to account for external sources of infection increases the statistical power to estimate the effect of antiviral prophylaxis.

A bias-reduced generalized estimating equation approach for proportional odds models with small-sample longitudinal ordinal data

BackgroundLongitudinal ordinal data are commonly analyzed using a marginal proportional odds model for relating ordinal outcomes to covariates in the biomedical and health sciences. The generalized estimating equation (GEE) consistently estimates the regression parameters of marginal models even if the working covariance structure is misspecified. For small-sample longitudinal binary data, recent studies have shown that the bias of regression parameters may result from the GEE and have addressed the issue by applying Firth’s adjustment for the likelihood score equation to the GEE as if generalized estimating functions were likelihood score functions. In this manuscript, for the proportional odds model for longitudinal ordinal data, the small-sample properties of the GEE were investigated, and a bias-reduced GEE (BR-GEE) was derived.MethodsBy applying the adjusted function originally derived for the likelihood score function of the proportional odds model to the GEE, we produced the BR-GEE. We investigated the small-sample properties of both GEE and BR-GEE through simulation and applied them to a clinical study dataset.ResultsIn simulation studies, the BR-GEE had a bias closer to zero, smaller root mean square error than the GEE with coverage probability of confidence interval near or above the nominal level. The simulation also showed that BR-GEE maintained a type I error rate near or below the nominal level.ConclusionsFor the analysis of longitudinal ordinal data involving a small number of subjects, the BR-GEE is advantageous for obtaining estimates of the regression parameters of marginal proportional odds models.

Conditional bias adjusted estimator of treatment effect in 2-in-1 adaptive design

ABSTRACT For implementation of adaptive design, the adjustment of bias in treatment effect estimation becomes an increasingly important topic in recent years. While adaptive design literature traditionally focuses on the control of type I error rate and the adjustment of overall unconditional bias, the research on adjusting conditional bias has been limited. This paper proposes a conditional bias adjustment estimator of treatment effect under the context of 2-in-1 adaptive design and aims to provide a comprehensive investigation on their statistical properties including bias, mean squared error and coverage probability of confidence intervals. It demonstrated that conditional bias adjusted estimators greatly reduce the conditional bias and have similarly negligible unconditional bias compared with mean and median (unconditional) unbiased estimators. In addition, the test statistics is constructed based on the conditional bias adjustment estimators and compared with the naive unadjusted test.

Are state-space stock assessment model confidence intervals accurate? Case studies with SAM and Barents Sea stocks

Our main contribution is to examine the reliability of confidence intervals using the SAM state-space fish stock assessment model used for the assessment of many stocks by the International Council for the Exploration of the Seas. We focus on frequentist statistical inferences and more specifically on inference conditioned on specific values of the state-space model random effects drawn from their process distribution. This is somewhat consistent with simulation self-test procedures that are commonly used to examine the reliability of state-space assessment model results. However, recent research has indicated that some estimation bias may be expected in the conditional setting. Hence, we also investigate recently proposed bias corrected confidence intervals appropriate for the conditional inference setting. The SAM simulation coverage probabilities of 95% confidence intervals for SSB and Fbar were usually slightly larger than 95%, but in a small number of years these coverage probabilities could be much smaller than 95%. The bias corrected confidence intervals were more reliable. When averaged over years, the SAM and bias corrected confidence interval coverage probabilities were similar for the Northeast Artic cod and saithe case studies, but the bias corrected confidence intervals performed much better overall for the haddock case study.

Bootstrap-t Confidence Interval on Local Polynomial Regression Prediction

In local polynomial regression, prediction confidence interval estimation using standard theory will give coverage probability close to exact coverage probability. However, if the normality assumption is not met, the bootstrap method makes it possible to apply it. The working principle of the bootstrap method uses the resampling method where the sample data becomes a population and there is no need to know the distribution of the sample data is normal or not. Indiscriminate selection of smoothing parameters allows scatterplot results from local polynomial regressions to be rough and can even lead to misleading statistical conclusions. It is necessary to consider the optimal smoothing parameters to get local polynomial regression predictions that are not overfitting or underfitting. We offer two new algorithms based on the nested bootstrap resampling method to determine the bootstrap-t confidence interval in predicting local polynomial regression. Both algorithms consider the search for optimal smoothing parameters. The first algorithm performs paired and residual bootstrap samples, and the second algorithm performs based on residuals with residuals. The first algorithm provides a scatterplot and reasonable coverage probability on relatively large sample data. In contrast, the second algorithm is more powerful for each data size, including for relatively small sample data sizes. The mean of the bootstrap-t confidence interval coverage probability shows that the second algorithm for second-degree local polynomial regression is better than the other three. However, the larger the sample data size gives, the closer the closer the average coverage probability of the two algorithms is to the nominal coverage probability.

Modified-Lindley distribution and its applications to the real data

In this paper, a new three-parameter lifetime distribution is proposed by mixing modified Weibull and generalized gamma distributions. The point estimation on the distribution parameters are discussed through several estimators. The interval estimation is also studied with two methods based on asymptotic normality and likelihood ratio. A Monte Carlo simulation study is performed to evaluate the biases and mean square errors behaviors of point estimates for a different sample of size. A simulation study is also conducted to investigate the coverage probabilities of confidence intervals. The distribution modeling analyses are provided based on several real data sets to demonstrate the fitting ability of the introduced distribution.

Performance of statistical methods to address treatment non-adherence in pragmatic clinical trials with point-treatment settings: a simulation study

Introduction: The instrumental variable (IV)-based methods (e.g., two-stage least square [2SLS], two-stage residual inclusion [2SRI], and nonparametric causal bound [NPCB]) can be used to address non-adherence in pragmatic trials. These methods require assumptions, e.g., exclusion restriction, although they are known to handle unmeasured confounding. The inverse probability-weighted per-protocol [IPW-PP] method is useful in the same setting but requires different assumptions (no unmeasured confounding). Although all these methods aim to address the same problem, comprehensive simulations to compare their performance are absent in the literature. We performed extensive simulations when (1) confounding is present, (2) confounder is unmeasured but exclusion restriction is met, (3) exclusion restriction is violated, and (4) non-adherence is one-sided and differential.
 Method: We compared the performance in terms of bias, standard error (SE), mean squared error (MSE), and 95% confidence interval coverage probability.
 Results: For setting-1, IPW-PP outperforms IV-methods in terms of bias, SE, MSE, and coverage for <80% non-adherence but produces high bias beyond that point. IPW-PP also has high biases, but 2SLS and 2SRI work well for setting-2. For setting-3, 2SLS and 2SRI perform the worst in all scenarios; IPW-PP produces unbiased estimates when necessary confounders are measured and adjusted. For setting-4, IPW-PP has less bias, but 2SLS and 2SRI have higher SE and MSE. NPCB has wider bounds in all scenarios. We also analyze a two-arm trial to estimate the effect of vitamin A supplementation on childhood mortality after addressing non-adherence.
 Conclusion: We need to be cautious using the IPW-PP when non-adherence is very high or strong unmeasured confounding and should avoid using the IV methods when the exclusion restriction assumption is violated or high differential non-adherence. Since assumptions are different and often untestable for IPW-PP and IV methods, we suggest analyzing data using both methods for a robust conclusion.

Generalized Inference on Stress-Strength Reliability in Generalized Pareto Model

The article focuses on the inference of stress-strength reliability in generalized Pareto model using the generalized variable approach and bootstrap percentile method. Simulation studies are conducted to obtain expected lengths and coverage probabilities of confidence intervals constructed using the generalized variable and the bootstrap percentile methods. An example consisting of real stress-strength data is also presented for illustrative purposes.

Practical Review and Comparison of Modified Covariance Estimators for Linear Mixed Models in Small‐sample Longitudinal Studies with Missing Data

SummaryMixed‐effects models for repeated measures (MMRMs) with an ‘unstructured’ (UN) covariance structure are frequently used in primary analyses for group comparisons of incomplete continuous longitudinal data from drug development trials. However, MMRM‐UN analysis could lead to convergence problems in numerical optimisation, especially in trials with a small sample size or high dropout rate. Although the so‐called sandwich covariance estimator is robust against the misspecification of the covariance structure, its performance deteriorates for small sample sizes. We review eight modified covariance estimators adjusted for small‐sample bias and compare their performances in the framework of MMRM analysis through simulations. In terms of the type 1 error rate and coverage probability of confidence intervals for group comparisons, Mancl and DeRouen's covariance estimator (MD) shows the best performance among the modified covariance estimators, followed by Fay and Graubard's estimator. The performance of MD is nearly equivalent to that of the Kenward–Roger method with a UN structure. The Kenward–Roger method with first‐order autoregressive structure results in substantial inflation of the type 1 error rate in the scenario where the variance of measurements increases across visits. In summary, we recommend the use of MD in MMRM analysis if the convergence problem involving a UN structure occurs in small clinical trials.

Correction to ‘Distribution‐free Approximate Methods for Constructing Confidence Intervals for Quantiles’

One way of evaluating the quality of a simulation study methodology is to estimate the Monte Carlo Error (MCE). The formula for this calculation was incorrectly implemented in Nagaraja & Nagaraja (2020). The correct formula that replaces equation (24) on page 96 of the original paper is as follows. The MCE was estimated for the confidence interval coverage probability calculations, and those results have been updated in Tables 29–56 of the Supporting Information. No conclusions changed due to this correction. Supporting info item Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Asymptotic distributions and performance of empirical skewness measures

A number of skewness measures have been proposed and applied to theoretical distributions. However, the corresponding empirical counterparts have been analyzed only rarely, especially with respect to their asymptotic properties and limit distributions. Six of these empirical measures are considered. After discussing some general properties, the limiting distribution for each measure is derived under weak assumptions. The performance of these estimators is analyzed in simulations using tests and the coverage probabilities of confidence intervals. A particular focus is put on the standardized central third moment as the most popular measure of skewness. Since it turns out to behave poorly, especially when sample sizes are small, the use of alternative and more suitable skewness measures is recommended. A real data application illustrates some of the findings.

The influence of nonnormality from primary studies on the standardized mean difference in meta-analysis.

In this study we investigated the influence of data nonnormality in the primary studies on meta-analysis of the standardized mean difference (SMD) for a two-independent-group design. The bias, mean squared error, and confidence interval coverage probability of the mean effect sizes under different types of population distributions were compared. Also, the performance of the Q test was examined. The results showed that oppositely skewed distributions (i.e., distributions skewed in different directions) showed poor performance for point and interval estimates of mean effect sizes in meta-analysis, especially when the tails were pointing toward each other. The previously found adverse impacts due to nonnormality in primary studies do not disappear when primary studies with nonnormal data are meta-analyzed, even when the average sample size and number of studies are large. The results also showed that, when the tails were pointing toward each other, the Type I error rates of the Q test were inflated. We suggest that the impact of violating the assumption of normality should not be ignored in meta-analysis.

Computational issues in fitting joint frailty models for recurrent events with an associated terminal event

Background and objective: Joint frailty regression models are intended for the analysis of recurrent event times in the presence of informative drop-outs. They have been proposed for clinical trials to estimate the effect of some treatment on the rate of recurrent heart failure hospitalisations in the presence of drop-outs due to cardiovascular death. Whereas a R-software-package for fitting joint frailty models is available, some technical issues have to be solved in order to use SASⓇ11SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. software, which is required in the regulatory environment of clinical trials.Methods: First, we demonstrate how to solve these issues by deriving proper likelihood-decompositions, in particular for the case of non-normally distributed random terms. Second, we perform a simulation study to evaluate the accuracy of different software-implementations (in SAS and R) in terms of convergence behavior, bias of model parameter estimates and coverage probabilities of confidence intervals. Therefore we developed SAS macros that facilitate the analysis and simulation of joint frailty data. These are provided as supplementary material along with comprehensive manuals.Results: Whereas estimates for regression coefficients are unbiased irrespective of the software, the bias of the remaining (nuisance) parameter estimates strongly depends on the software: SAS is shown to be much more efficient in avoiding bias compared to R. However, even in SAS a careful choice of the implementation is required to get reliable results, in particular for the joint gamma frailty model. By far the best performance is reached with a SAS-implementation that makes use of the probability integral transformation method.Conclusions: We have shown, that getting reliable results from joint frailty models is not straightforward and users should be aware about the computational options between and within software packages. Based on our simulation study, we elaborate recommendations on these options. In addition, our provided SAS macros may encourage statistical practitioners to apply these models in clinical trials with recurrent event data and potentially informative drop-outs.

Estimating population size with imperfect detection using a parametric bootstrap

We develop a novel method of estimating population size from imperfectly detected counts of individuals and a separate estimate of detection probability. Observed counts are separated into classes within which detection probability is assumed constant. Within a detection class, counts are modeled as a single binomial observation X with success probability p where the goal is to estimate index N. We use a Horvitz–Thompson‐like estimator for N and account for uncertainty in both sample data and estimated success probability via a parametric bootstrap. Unlike capture–recapture methods, our model does not require repeated sampling of the population. Our method is able to achieve good results, even with small X. We show in a factorial simulation study that the median of the bootstrapped sample has small bias relative to N and that coverage probabilities of confidence intervals for N are near nominal under a wide array of scenarios. Our methodology begins to break down when P(X=0)>0.1 but is still capable of obtaining reasonable confidence coverage. We illustrate the proposed technique by estimating (1) the size of a moose population in Alaska and (2) the number of bat fatalities at a wind power facility, both from samples with imperfect detection probabilities, estimated independently.

Confidence, prediction, and tolerance in linear mixed models.

The literature about Prediction Interval (PI) and Tolerance Interval (TI) in linear mixed models is usually developed for specific designs, which is a main limitation to their use. This paper proposes to reformulate the two‐sided PI to be generalizable under a wide variety of designs (one random factor, nested and crossed designs for multiple random factors, and balanced or unbalanced designs). This new methodology is based on the Hessian matrix, namely, the inverse of (observed) Fisher Information matrix, and is built with a cell mean model. The degrees of freedom for the total variance are calculated with the generalized Satterthwaite method and compared to the Kenward‐Roger's degrees of freedom for fixed effects. Construction of two‐sided TIs are also detailed with one random factor, and two nested and two crossed random variables. An extensive simulation study is carried out to compare the widths and coverage probabilities of Confidence Intervals (CI), PIs, and TIs to their nominal levels. It shows excellent coverage whatever the design and the sample size are. Finally, these CIs, PIs, and TIs are applied to two real data sets: one from orthopedic surgery study (intralesional resection risk) and the other from assay validation study during vaccine development.

Nonparametric estimation of the cumulative incidence function under outcome misclassification using external validation data.

Misclassification of outcomes or event types is common in health sciences research and can lead to serious bias when estimating the cumulative incidence functions in settings with competing risks. Recent work has shown how to estimate nonparametric cumulative incidence functions in the presence of nondifferential outcome misclassification when the misclassification probabilities are known. Here, we extend this approach to account for misclassification that is differential with respect to important predictors of the outcome using misclassification probabilities estimated from external validation data. Moreover, we propose a bootstrap approach in which the observations from both the main study data and the external validation study are resampled to allow the uncertainty in the misclassification probabilities to propagate through the analysis into the final confidence intervals, ensuring appropriate confidence interval coverage probabilities. The proposed estimator is shown to be uniformly consistent and simulation studies indicate that both the estimator and the standard error estimation approach perform well in finite samples. The methodology is applied to estimate the cumulative incidence of death and disengagement from HIV care in a large cohort of HIV infected individuals in sub-Saharan Africa, where a significant death underreporting issue leads to outcome misclassification. This analysis uses external validation data from a separate study conducted in the same country.

On mean-based bivariate Birnbaum-Saunders distributions: Properties, inference and application

Birnbaum-Saunders models have been widely used to describe data following positive-skew distributions. In this article, we introduce a bivariate Birnbaum-Saunders distribution which has the mean as one of its parameters, allowing us to mimic the standard parameterization of the normal distribution, but in an asymmetric framework. We derive some properties of the mean-based bivariate Birnbaum-Saunders distribution useful for statistical and reliability analyses. Maximum likelihood and modified moment estimations of the model parameters and associated inference are considered. A simulation study is conducted to evaluate the performance of the corresponding estimators and coverage probabilities of confidence intervals are also discussed. Finally, a real-world data analysis is carried out for illustrating the potential of the proposed model.

A unified method for improved inference in random effects meta-analysis.

Random effects meta-analyses have been widely applied in evidence synthesis for various types of medical studies. However, standard inference methods (e.g. restricted maximum likelihood estimation) usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations; for instance, coverage probabilities of confidence intervals can be substantially below the nominal level. The main reason is that these inference methods rely on large sample approximations even though the number of synthesized studies is usually small or moderate in practice. In this article, we solve this problem using a unified inference method based on Monte Carlo conditioning for broad application to random effects meta-analysis. The developed method provides improved confidence intervals with coverage probabilities that are closer to the nominal level than standard methods. As specific applications, we provide new inference procedures for three types of meta-analysis: conventional univariate meta-analysis for pairwise treatment comparisons, meta-analysis of diagnostic test accuracy, and multiple treatment comparisons via network meta-analysis. We also illustrate the practical effectiveness of these methods via real data applications and simulation studies.

The large sample coverage probability of confidence intervals in general regression models after a preliminary hypothesis test

AbstractWe derive a computationally convenient formula for the large sample coverage probability of a confidence interval for a scalar parameter of interest following a preliminary hypothesis test that a specified vector parameter takes a given value in a general regression model. Previously, this large sample coverage probability could only be estimated by simulation. Our formula only requires the evaluation, by numerical integration, of either a double or a triple integral, irrespective of the dimension of this specified vector parameter. We illustrate the application of this formula to a confidence interval for the odds ratio of myocardial infarction when the exposure is recent oral contraceptive use, following a preliminary test where two specified interactions in a logistic regression model are zero. For this real‐life data, we compare this large sample coverage probability with the actual coverage probability of this confidence interval, obtained by simulation.