Empirical Coverage Probabilities Research Articles

Testing distribution for multiplicative distortion measurement errors

. In this article, we study a goodness of fit test for a multiplicative distortion model under a uniformly distributed but unobserved random variable. The unobservable variable is distorted in a multiplicative fashion by an observed confounding variable. The proposed k-th power test statistic is based on logarithmic transformed observations and a correlation coefficient-based estimator without distortion measurement errors. The proper choice of k is discussed through the empirical coverage probabilities. The asymptotic null distribution of the test statistics are obtained with known asymptotic variances. Next, we proposed the conditional mean calibrated test statistic when a variable is distorted in a multiplicative fashion. We conduct Monte Carlo simulation experiments to examine the performance of the proposed test statistics.

Sample size determination for interval estimation of the prevalence of a sensitive attribute under non-randomized response models.

A sufficient number of participants should be included to adequately address the research interest in the surveys with sensitive questions. In this paper, sample size formulas/iterative algorithms are developed from the perspective of controlling the confidence interval width of the prevalence of a sensitive attribute under four non-randomized response models: the crosswise model, parallel model, Poisson item count technique model and negative binomial item count technique model. In contrast to the conventional approach for sample size determination, our sample size formulas/algorithms explicitly incorporate an assurance probability of controlling the width of a confidence interval within the pre-specified range. The performance of the proposed methods is evaluated with respect to the empirical coverage probability, empirical assurance probability and confidence width. Simulation results show that all formulas/algorithms are effective and hence are recommended for practical applications. A real example is used to illustrate the proposed methods.

LIMBARE: An Advanced Linear Mixed-Effects Breakpoint Analysis With Robust Estimation Method With Applications to Longitudinal Ophthalmic Studies.

Broken stick analysis is a widely used approach for detecting unknown breakpoints where the association between measurements is nonlinear. We propose LIMBARE, an advanced linear mixed-effects breakpoint analysis with robust estimation, especially designed for longitudinal ophthalmic studies. LIMBARE accommodates repeated measurements from both eyes and over time, and it effectively addresses the presence of outliers. The model setup of LIMBARE and the computing algorithm for point and confidence interval estimates of the breakpoint were introduced. The performance of LIMBARE and other competing methods was assessed via comprehensive simulation studies and application to a longitudinal ophthalmic study with 216 eyes (145 subjects) followed for an average of 3.7 ± 1.3years to examine the longitudinal association between structural and functional measurements. In simulation studies, LIMBARE showed the smallest bias and mean squared error for estimating the breakpoint, with an empirical coverage probability of corresponding confidence interval estimates closest to the nominal level for scenarios with and without outlier data points. In the application to the longitudinal ophthalmic study, LIMBARE detected two breakpoints between visual field mean deviation (MD) and retinal nerve fiber layer thickness and one breakpoint between MD and cup-to-disc ratio, whereas the cross-sectional analysis approach detected only one and none, respectively. LIMBARE enhances breakpoint estimation accuracy in longitudinal ophthalmic studies, and the cross-sectional analysis approach is not recommended for future studies. Our proposed method and companion R package provide a valuable computational tool for advancing longitudinal ophthalmology research and exploring the association relationships among ophthalmic variables.

Interval estimation of relative risks for combined unilateral and bilateral correlated data

ABSTRACT Measurements are generally collected as unilateral or bilateral data in clinical trials, epidemiology, or observational studies. For example, in ophthalmology studies, the primary outcome is often obtained from one eye or both eyes of an individual. In medical studies, the relative risk is usually the parameter of interest and is commonly used. In this article, we develop three confidence intervals for the relative risk for combined unilateral and bilateral correlated data under the equal dependence assumption. The proposed confidence intervals are based on maximum likelihood estimates of parameters derived using the Fisher scoring method. Simulation studies are conducted to evaluate the performance of proposed confidence intervals with respect to the empirical coverage probability, the mean interval width, and the ratio of mesial non-coverage probability to the distal non-coverage probability. We also compare the proposed methods with the confidence interval based on the method of variance estimates recovery and the confidence interval obtained from the modified Poisson regression model with correlated binary data. We recommend the score confidence interval for general applications because it best controls converge probabilities at the 95% level with reasonable mean interval width. We illustrate the methods with a real-world example.

Predicting areal extent of groundwater contamination through geostatistical methods exploration in a data-limited rural basin

The areal extent of regulatory standard exceedance for groundwater arsenic, fluoride, and nitrate was estimated and compared for multiple methods to evaluate groundwater contamination in a rural basin of New Mexico, USA. First, three geostatistical kriging techniques – Ordinary Kriging (OK), Universal Kriging (UK), and Empirical Bayesian Kriging (EBK) – were explored. Next, the exploration was extended to multivariate analysis (Co-OK, Co-UK, and EBK-regression) using groundwater pH as the other variable, compared to interpolation only considering the contaminant as a primary variable. Finally, non-parametric Indicator Kriging (IK) was utilized for probability mapping and assessing the likelihood of surpassing critical thresholds for contaminant concentrations. The contamination area changes with different kriging methods deployed, and the performances of kriging variants vary for the three chemical components. The models that best fit the empirical data were determined. The study examined the influence of the explanatory variable on the multivariate kriging prediction accuracy. The multivariate kriging approaches enhanced surface prediction accuracy and tended to cover the data limitation of the primary variable interpolations. The multivariate kriging improved spatial prediction of arsenic and fluoride by increasing the Empirical Coverage Probability (ECP) up to 0.05–0.25 from case to case; however, no improvement was observed in ECP for nitrate contamination distribution, as the explanatory variable pH was only weakly correlated with nitrate, suggesting that a stronger correlation between the primary and secondary variables can increase the probability of obtaining prediction improvements through multivariate kriging. This study developed a methodology for characterizing and identifying locations where groundwater had elevated concentrations of contaminants, threatening human health through drinking water exposure. By improving the spatial characterization of groundwater contaminants within the data-limited rural basin, policymakers planning improvements will support mitigation strategies to manage contamination sources of groundwater resources.

Revisiting sample size planning for receiver operating characteristic studies: A confidence interval approach with precision and assurance.

Estimation of areas under receiver operating characteristic curves and their differences is a key task in diagnostic studies. Here we develop closed-form sample size formulas for such studies with a focus on estimation rather than hypothesis testing, by explicitly incorporating pre-specified precision and assurance, with precision denoted by the lower limit of confidence interval and assurance denoted by the probability of achieving that lower limit. For sample size estimation purposes, we introduce a normality-based variance function for valid estimation allowing for unequal variances of observations in the disease and non-disease groups. Simulation results demonstrate that the proposed formulas produce empirical assurance probability close to the pre-specified assurance probability and empirical coverage probability close to the nominal level. Compared with a frequently used existing variance function, the proposed function provides more accurate and efficient sample size estimates. For an illustration of the proposed formulas, we present real-world worked examples. To facilitate implementation, we have developed an online calculator openly available at https://dishu.page/calculator/.

Conformal asymmetric multi-quantile generative transformer for day-ahead wind power interval prediction

With the rapid increase in the installed capacity of wind power, day-ahead wind power interval prediction is becoming more and more important. To solve such a challenging problem and provide intervals of higher quality, this paper proposes a prediction method based on conformal asymmetric multi-quantile generative transformer. Herein, the multi-quantile generative transformer is a deep learning model that generates multi-quantile forecast results for next day through one forward propagation process. Then, two quantiles, whose width is the smallest while satisfying the nominal confidence constraint, are selected from the predicted sequence as the upper bound and lower bound of the asymmetric interval. Furthermore, we introduce the conformal quantile regression to calibrate the bounds of the prediction interval to ensure that its coverage rate is as close as nominal confidence. The experiments show that the proposed method surpasses the benchmarks by providing narrower prediction intervals with more accurate empirical coverage probability. Under nominal confidence 90%, it gives prediction intervals with average empirical coverage probability of 90.50% and normalized average width of 0.44 on four wind farms. Compared with symmetric prediction intervals given by common benchmark quantile long short term memory network, the average width is reduced by 19.6%.

Adjusting for publication bias in meta-analysis via inverse probability weighting using clinical trial registries.

Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models, and they have some advantages over the widely used trim-and-fill bias-correction method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals, or require a rather complicated sensitivity analysis process. Herein, we develop a simple publication bias adjustment method by utilizing the information on conducted but still unpublished trials from clinical trial registries. We introduce an estimating equationfor parameter estimation in the selection function by regarding the publication bias issue as a missing data problem under the missing not at random assumption. With the estimated selection function, we introduce the inverse probability weighting (IPW) method to estimate the overall mean across studies. Furthermore, the IPW versions of heterogeneity measures such as the between-study variance and the I2 measure are proposed. We propose methods to construct confidence intervals based on asymptotic normal approximation as well as on parametric bootstrap. Through numerical experiments, we observed that the estimators successfully eliminated bias, and the confidence intervals had empirical coverage probabilities close to the nominal level. On the other hand, the confidence interval based on asymptotic normal approximation is much wider in some scenarios than the bootstrap confidence interval. Therefore, the latter is recommended for practicaluse.

Comparison of confidence intervals for variance components in an unbalanced one-way random effects model

Abstract The purpose of this paper is to study and compare the methods for constructing confidence intervals for variance components in an unbalanced one-way random effects model. The methods are based on a classical exact, generalised pivotal quantity, a fiducial inference and a fiducial generalised pivotal quantity. The comparison of criteria involves the empirical coverage probability that maintains at the nominal confidence level of 0.95 and the shortest average length of the confidence interval. The simulation results show that the method based on the generalised pivotal quantity and the fiducial inference perform very well in terms of both the empirical coverage probability and the average length of the confidence interval. The classical exact method performs well in some situations, while the fiducial generalised pivotal quantity performs well in a very unbalanced design. Therefore, the method based on the generalised pivotal quantity is recommended for all situations.

ISO Linear Calibration and Measurement Uncertainty of the Result Obtained With the Calibrated Instrument

Abstract We address the problem of linear comparative calibration, a special case of linear calibration where both variables are measured with errors, and the analysis of the uncertainty of the measurement results obtained with the calibrated instrument. The concept is explained in detail using the calibration experiment of the pressure transducer and the subsequent analysis of the measurement uncertainties. In this context, the calibration and the measurements with the calibrated instrument are performed according to ISO Technical Specification 28037:2010 (here referred to as ISO linear calibration), based on the approximate linear calibration model and the application of the law of propagation of uncertainty (LPU) in this approximate model. Alternatively, estimates of the calibration line parameters, their standard uncertainties, the coverage intervals and the associated probability distributions are obtained using the Monte Carlo method (MCM) based on the law of propagation of distributions (LPD). Here we also obtain the probability distributions and the coverage interval for the quantities measured with the calibrated instrument. Furthermore, motivated by the model structure of this particular example, we conducted a simulation study that presents the empirical coverage probabilities of the ISO and MCM coverage intervals and investigates the influence of the sample size, i.e. the number of calibration points in the measurement range, and the different combinations of measurement uncertainties. The study generally confirms the good properties and validity of the ISO technical specification within the considered (limited) framework of experimental designs motivated by real-world application, with small uncertainties in relation to the measurement range. We also point out the potential weaknesses of this method that require increased user attention and emphasise the need for further research in this area.

Sieve bootstrapping the memory parameter in long-range dependent stationary functional time series

We consider a sieve bootstrap procedure to quantify the estimation uncertainty of long-memory parameters in stationary functional time series. We use a semiparametric local Whittle estimator to estimate the long-memory parameter. In the local Whittle estimator, discrete Fourier transform and periodogram are constructed from the first set of principal component scores via a functional principal component analysis. The sieve bootstrap procedure uses a general vector autoregressive representation of the estimated principal component scores. It generates bootstrap replicates that adequately mimic the dependence structure of the underlying stationary process. We first compute the estimated first set of principal component scores for each bootstrap replicate and then apply the semiparametric local Whittle estimator to estimate the memory parameter. By taking quantiles of the estimated memory parameters from these bootstrap replicates, we can nonparametrically construct confidence intervals of the long-memory parameter. As measured by coverage probability differences between the empirical and nominal coverage probabilities at three levels of significance, we demonstrate the advantage of using the sieve bootstrap compared to the asymptotic confidence intervals based on normality.

Sample Size Determination for Interval Estimation of the Prevalence of a Sensitive Attribute Under Randomized Response Models

Studies with sensitive questions should include a sufficient number of respondents to adequately address the research interest. While studies with an inadequate number of respondents may not yield significant conclusions, studies with an excess of respondents become wasteful of investigators’ budget. Therefore, it is an important step in survey sampling to determine the required number of participants. In this article, we derive sample size formulas based on confidence interval estimation of prevalence for four randomized response models, namely, the Warner’s randomized response model, unrelated question model, item count technique model and cheater detection model. Specifically, our sample size formulas control, with a given assurance probability, the width of a confidence interval within the planned range. Simulation results demonstrate that all formulas are accurate in terms of empirical coverage probabilities and empirical assurance probabilities. All formulas are illustrated using a real-life application about the use of unethical tactics in negotiation.

Modelling the downside risk potential of mutual fund returns

Investors are becoming more sensitive about returns and losses, especially when the investments are exposed to downside risk potential in the financial markets. Despite the computational intensity of the downside risk measures, they are very widely applied to construct a portfolio and evaluate performance in terms of the investors’ loss aversion. Value-at-risk (VaR) has emerged as an industry standard to analyze the market downside risk potential. The approaches used to measure VaR vary from the standard approaches to more recently introduced highly sophisticated volatility models. In this paper, the standard approaches (student-t-distribution, log normal, historical simulation) and sophisticated volatility models (EWMA, GARCH (1,1)) both have been used to estimate the VaR of mutual funds in the Saudi Stock Exchange between June 2017 and June 2020. The VaR approaches have been subjected to conditional coverage backtest to identify the model that is the best at predicting VaR. The empirical coverage probability of the models reveals that EWMA was able to capture VaR better than the other models at a higher significance level followed by GARCH (1,1).

Bayesian parameter estimation in chiral effective field theory using the Hamiltonian Monte Carlo method

The number of low-energy constants (LECs) in chiral effective field theory ($\ensuremath{\chi}\mathrm{EFT}$) grows rapidly with increasing chiral order, necessitating the use of Markov chain Monte Carlo techniques for sampling their posterior probability density function. For this we introduce a Hamiltonian Monte Carlo (HMC) algorithm and sample the LEC posterior up to next-to-next-to-leading order (NNLO) in the two-nucleon sector of $\ensuremath{\chi}\mathrm{EFT}$. We find that the sampling efficiency of HMC is three to six times higher compared to an affine-invariant sampling algorithm. We analyze the empirical coverage probability and validate that the NNLO model yields predictions for two-nucleon scattering data with largely reliable credible intervals, provided that one ignores the leading-order EFT expansion parameter when inferring the variance of the truncation error. We also find that the NNLO truncation error dominates the error budget.

Confidence Intervals for Assessing Non-Inferiority with Assay Sensitivity in a Three-Arm Trial with Normally Distributed Endpoints

Various approaches including hypothesis test and confidence interval (CI) construction have been proposed to assess non-inferiority and assay sensitivity via a known fraction or pre-specified margin in three-arm trials with continuous or discrete endpoints. However, there is little work done on the construction of the non-inferiority margin from historical data and simultaneous generalized CIs (SGCIs) in a three-arm trial with the normally distributed endpoints. Based on the generalized fiducial method and the square-and-add method, we propose two simultaneous CIs for assessing non-inferiority and assay sensitivity in a three-arm trial. For comparison, we also consider the Wald-type Bonferroni simultaneous CI and parametric bootstrap simultaneous CI. An algorithm for evaluating the optimal sample size for attaining the pre-specified power is given. Simulation studies are conducted to investigate the performance of the proposed CIs in terms of their empirical coverage probabilities. An example taken from the mildly asthmatic study is illustrated using the proposed simultaneous CIs. Empirical results show that the proposed generalized fiducial method and the square-and-add method behave better than other two compared CIs.

Prediction intervals of the COVID-19 cases by HAR models with growth rates and vaccination rates in top eight affected countries: Bootstrap improvement

This paper is devoted to modeling and predicting COVID-19 confirmed cases through a multiple linear regression. Especially, prediction intervals of the COVID-19 cases are extensively studied. Due to long-memory feature of the COVID-19 data, a heterogeneous autoregression (HAR) is adopted with Growth rates and Vaccination rates; it is called HAR-G-V model. Top eight affected countries are taken with their daily confirmed cases and vaccination rates. Model criteria results such as root mean square error (RMSE), mean absolute error (MAE), R2, AIC and BIC are reported in the HAR models with/without the two rates. The HAR-G-V model performs better than other HAR models. Out-of-sample forecasting by the HAR-G-V model is conducted. Forecast accuracy measures such as RMSE, MAE, mean absolute percentage error and root relative square error are computed. Furthermore, three types of prediction intervals are constructed by approximating residuals to normal and Laplace distributions, as well as by employing bootstrap procedure. Empirical coverage probability, average length and mean interval score are evaluated for the three prediction intervals. This work contributes three folds: a novel trial to combine both growth rates and vaccination rates in modeling COVID-19; construction and comparison of three types of prediction intervals; and an attempt to improve coverage probability and mean interval score of prediction intervals via bootstrap technique.

Time-specific average estimation of dynamic panel regressions.

This paper introduces an unbiased estimator based on least squares involving time-specific cross-sectional averages for a first-order panel autoregression with a strictly exogenous covariate. The proposed estimator is straightforward to implement as long as the variables of interest have sufficient time variation. The number of cross-sections (N) and the number of time periods (T) can be large, and there is no restriction on the growth rate of N relative to T. It is demonstrated via both theory and a simulation study that the estimator is asymptotically unbiased, and it can provide correct empirical coverage probabilities for the 'true' coefficients of the model for various combinations of N and T. An empirical application is also provided to confirm the feasibility of the proposed approach.

Confidence intervals for assessing equivalence of two treatments with combined unilateral and bilateral data

Responses from the paired organs are generally highly correlated in bilateral studies, statistical procedures ignoring the correlation could lead to incorrect results. Note the intraclass correlation in the study of combined unilateral and bilateral outcomes; 11 confidence intervals (CIs) including 7 asymptotic CIs and 4 Bootstrap-resampling CIs for assessing the equivalence of 2 treatments are derived under Rosner's correlated binary data model. Performance is evaluated with respect to the empirical coverage probability (ECP), the empirical coverage width (ECW) and the ratio of the mesial non-coverage probability to the non-coverage probability (RMNCP) via simulation studies. Simulation results show that (i) all CIs except for the Wald CI and the bias-corrected Bootstrap percentile CI generally produce satisfactory ECPs and hence are recommended; (ii) all CIs except for the bias-corrected Bootstrap percentile CI provide preferred RMNCPs and are more symmetrical; (iii) as the measurement of the dependence increases, the ECWs of all CIs except for the score CI and the profile likelihood CI show increasing patterns that look like linear, while there is no obvious pattern on the ECPs of all CIs except for the profile likelihood CI. A data set from an otolaryngologic study is used to illustrate the proposed methods.

Confidence intervals of proportion differences for stratified combined unilateral and bilateral data

In otolaryngologic studies, subjects may contribute either unilateral observations from only one ear or bilateral observations from a pair of ears. For bilateral subjects, each person contributes information from two ears, the values of which are generally highly correlated. To avoid the confounding effect in stratified otolaryngologic studies, stratified data analysis is an important research topic. Based on the dependence model in stratified designs, this article presents five simultaneous confidence intervals (CIs) and two bootstrap simultaneous CIs for proportion differences with combined unilateral and bilateral data. Six approximate CIs and two bootstrap CIs for the common proportion difference are also developed. The proposed CIs are evaluated by empirical coverage probability, empirical coverage width and the ratio of mesial non-coverage probability to the non-coverage probability. Simulation results show that the simultaneous CIs based on Wilson method, the inverse hyperbolic tangent transformation, score statistic and the bootstrap percentile simultaneous CIs perform well and hence be recommended for applications, score CI and the CI based on Cochran statistic for the common proportion difference behave well even under small sample sizes, other CIs produce good performance when sample size is not small. A real otolaryngology study data is used to illustrate the proposed methodologies.

Assessing the variability of posterior probabilities in Gaussian model-based clustering

We propose a variant of the bootstrap to assess the variability of posterior probabilities arising from Gaussian model-based clustering. The bootstrap variant uses predictions based on out-of-bootstrap-sample observations and then constructs confidence intervals for the posterior probabilities using the percentile method. The methodology outperforms the multivariate Delta method approach when comparing empirical coverage probabilities on simulated data. The proposed and multivariate Delta methods are also illustrated on the well-known Iris data set.