Nominal Coverage Rate Research Articles

Efficient analysis of split-plot experimental designs using model averaging

Split-plot experimental data are often analyzed as if the data came from a completely randomized design. As is well known, ignoring the different levels of randomization and replication can lead to serious inferential errors. However, in some experiments, including many of the ocean global change experiments that motivated this research, variation between whole-plot experimental units may be small relative to variation between subplot units. Even though a factorial analysis will often perform poorly in general, in this special case it outperforms a split-plot analysis, providing narrower confidence intervals for treatment means and differences with coverage rates close to the desired level. The performance of the proposed model-averaged analysis was compared to a classical split-plot analysis via a simulation study, and its utility demonstrated for an ocean global change experiment examining growth and condition of a juvenile mussel species. In our simulation study, model-averaged confidence intervals for whole-plot treatment means or comparisons of means were up to 40% narrower than split-plot confidence intervals while maintaining close to nominal coverage rates. In our example experiment, we observed narrowing of up to 25%. We recommend model averaging as a preferred approach when variation between whole-plot experimental units is expected to be less than between subplot units, with a few caveats for studies with very few replicates.

Solar GHI Ensemble Prediction Based on a Meteorological Model and Method Kalman Filter

The intensity of light emanating from sun is determined by using a meteorological version and is altered with the numerical version, and the forecast accuracy is improved in advance by using Kalman Filter. As the accuracy of the version output related to its specific position is often questionable, group prediction constituting three members is suggested and agreed upon measurement. Also, this ensemble prediction provides an estimation of the solar global horizontal irradiance uncertainty (i.e., coverage rate of the prediction interval), which can be useful to provide flexible energy production forecasts. This article displays how the method Kalman filter could be used as an error correction way to alter the predicted irradiance value. The Kalman filter ameliorates the prediction of solar global horizontal irradiance as well as its interval. As the empirical coverage rate increases and closes to the nominal coverage rate, the interval size reduces.

Bayesian Inference for Estimating Subset Proportions using Differentially Private Counts

Abstract Recently, several organizations have considered using differentially private algorithms for disclosure limitation when releasing count data. The typical approach is to add random noise to the counts sampled from, for example, a Laplace distribution or symmetric geometric distribution. One advantage of this approach, at least for some differentially private algorithms, is that analysts know the noise distribution and hence have the opportunity to account for it when making inferences about the true counts. In this article, we present Bayesian inference procedures to estimate the posterior distribution of a subset proportion, that is, a ratio of two counts, given the released values. We illustrate the methods under several scenarios, including when the released counts come from surveys or censuses. Using simulations, we show that the Bayesian procedures can result in accurate inferences with close to nominal coverage rates.

Phylogenetic multilevel meta‐analysis: A simulation study on the importance of modelling the phylogeny

Abstract Meta‐analyses in ecology and evolution require special attention due to certain study characteristics in these fields. First, the primary articles in these fields usually report results that are observed from studies conducted with different species, and the phylogeny among the species violates the independence assumption. Second, articles frequently allow the computation of multiple effect sizes which cannot be accounted for by conventional meta‐analytic models. While both issues can be dealt with by utilizing a multilevel model that accounts for phylogeny, the performance of such a model has not been examined extensively. In this article, we investigate the performance of this model in comparison with some simpler models. We conducted an extensive simulation study where data with different hierarchical structures (in terms of study and species levels) were generated and then various models were fitted to examine their performance. The models we used include the conventional random effects and multilevel random‐effects models along with more complex multilevel models that account for species‐level variance with different variance components. Furthermore, we present an illustrative application of these models based on the data from a meta‐analysis on sizeassortative mating and comment on the results in light of the findings from the simulation study. Our simulation results show that, when the phylogenetic relationships among the species are at least moderately strong, only the most complex model that decomposes the species‐level variance into nonphylogenetic and phylogenetic components provides approximately unbiased estimates of the overall mean and variance components and yields confidence intervals with an approximately nominal coverage rate. Contrarily, removing the phylogenetic or non‐phylogenetic component leads to biased variance component estimates and an increased risk for incorrect inferences about the overall mean. These findings are supported by the results derived from the illustrative application. Based on our results, we suggest that meta‐analyses in ecology and evolution should use the model that accounts for both the nonphylogenetic and phylogenetic species‐level variance in addition to the multilevel structure of the data. Any attempts to simplify this model, such as using only the phylogenetic variance component, may lead to erroneous inferences from the data.

Battery state of health modeling and remaining useful life prediction through time series model

While most existing degradation modeling methods for rechargeable batteries consider a deterministic degradation model such as exponential model, this paper presents a time series model for battery degradation paths resembling experimental data on cycle aging. This model is based on breaking down the degradation path into segments by fitting a multiple-change-point linear model, which accounts for the degradation structure by regressing the segment lengths and the slope changes. These two variables are modeled by two sub-models: an autoregressive model with covariates for the slope changes at the change points and a survival regression model for the segment lengths that allows for censored data caused by interruptions during battery cycling. The combined model is able to predict a full battery degradation path based on historical paths, and predict the remaining degradation path even based merely on the partial path. The proposed model can also be used to produce confidence intervals for battery’s useful life by applying the method of parametric bootstrap to generate the empirical bootstrap distribution. The application of the proposed model is demonstrated with data from lithium iron phosphate and lithium nickel manganese cobalt oxide batteries. The comparison on prediction mean between proposed model, deterministic models with particle filter and recurrent neural network shows that the proposed model can make better prediction when capacity plunge is not present. The validation with simulations shows that the proposed model is reliable when complete historical paths are available as the simulation coverage rates are close to the nominal coverage rate 90%.

Model Averaging in Viral Dynamic Models.

The paucity of experimental data makes both inference and prediction particularly challenging in viral dynamic models. In the presence of several candidate models, a common strategy is model selection (MS), in which models are fitted to the data but only results obtained with the "best model" are presented. However, this approach ignores model uncertainty, which may lead to inaccurate predictions. When several models provide a good fit to the data, another approach is model averaging (MA) that weights the predictions of each model according to its consistency to the data. Here, we evaluated by simulations in a nonlinear mixed-effect model framework the performances of MS and MA in two realistic cases of acute viral infection, i.e., (1) inference in the presence of poorly identifiable parameters, namely, initial viral inoculum and eclipse phase duration, (2) uncertainty on the mechanisms of action of the immune response. MS was associated in some scenarios with a large rate of false selection. This led to a coverage rate lower than the nominal coverage rate of 0.95 in the majority of cases and below 0.50 in some scenarios. In contrast, MA provided better estimation of parameter uncertainty, with coverage rates ranging from 0.72 to 0.98 and mostly comprised within the nominal coverage rate. Finally, MA provided similar predictions than those obtained with MS. In conclusion, parameter estimates obtained with MS should be taken with caution, especially when several models well describe the data. In this situation, MA has better performances and could be performed to account for model uncertainty.

A Pan-Cancer and Polygenic Bayesian Hierarchical Model for the Effect of Somatic Mutations on Survival.

We built a novel Bayesian hierarchical survival model based on the somatic mutation profile of patients across 50 genes and 27 cancer types. The pan-cancer quality allows for the model to “borrow” information across cancer types, motivated by the assumption that similar mutation profiles may have similar (but not necessarily identical) effects on survival across different tissues of origin or tumor types. The effect of a mutation at each gene was allowed to vary by cancer type, whereas the mean effect of each gene was shared across cancers. Within this framework, we considered 4 parametric survival models (normal, log-normal, exponential, and Weibull), and we compared their performance via a cross-validation approach in which we fit each model on training data and estimate the log-posterior predictive likelihood on test data. The log-normal model gave the best fit, and we investigated the partial effect of each gene on survival via a forward selection procedure. Through this we determined that mutations at TP53 and FAT4 were together the most useful for predicting patient survival. We validated the model via simulation to ensure that our algorithm for posterior computation gave nominal coverage rates. The code used for this analysis can be found at https://github.com/sarahsamorodnitsky/Pan-Cancer-Survival-Modeling.git, and the results are summarized at http://ericfrazerlock.com/surv_figs/SurvivalDisplay.html.

A Pan-Cancer and Polygenic Bayesian Hierarchical Model for the Effect of Somatic Mutations on Survival

We built a novel Bayesian hierarchical survival model based on the somatic mutation profile of patients across 50 genes and 27 cancer types. The pan-cancer quality allows for the model to borrow information across cancer types, motivated by the assumption that similar mutation profiles may have similar (but not necessarily identical) effects on survival across different tissues-of-origin or tumor types. The effect of a mutation at each gene was allowed to vary by cancer type while the mean effect of each gene was shared across cancers. Within this framework we considered four parametric survival models (normal, log-normal, exponential, and Weibull), and we compared their performance via a cross-validation approach in which we fit each model on training data and estimate the log-posterior predictive likelihood on test data. The log-normal model gave the best fit, and we investigated the partial effect of each gene on survival via a forward selection procedure. Through this we determined that mutations at TP53 and FAT4 were together the most useful for predicting patient survival. We validated the model via simulation to ensure that our algorithm for posterior computation gave nominal coverage rates. The code used for this analysis can be found at this http URL , and the results are at this http URL .

Exact Adaptive Confidence Intervals for Small Areas

Abstract In the analysis of survey data, it is of interest to estimate and quantify uncertainty about means or totals for each of several nonoverlapping subpopulations or areas. When the sample size for a given area is small, standard confidence intervals based on data only from that area can be unacceptably wide. In order to reduce interval width, practitioners often utilize multilevel models in order to borrow information across areas, resulting in intervals centered around shrinkage estimators. However, such intervals only have the nominal coverage rate on average across areas under the assumed model for across-area heterogeneity. The coverage rate for a given area depends on the actual value of the area mean and can be nearly zero for areas with means that are far from the across-group average. As such, the use of uncertainty intervals centered around shrinkage estimators are inappropriate when area-specific coverage rates are desired. In this article, we propose an alternative confidence interval procedure for area means and totals under normally distributed sampling errors. This procedure not only has constant 1−α frequentist coverage for all values of the target quantity but also uses auxiliary information to borrow information across areas. Because of this, the corresponding intervals have shorter expected lengths than standard confidence intervals centered on the unbiased direct estimator. Importantly, the coverage of the procedure does not depend on the assumed model for across-area heterogeneity. Rather, improvements to the model for across-area heterogeneity result in reduced expected interval width.

Statistical properties of methods based on the Q-statistic for constructing a confidence interval for the between-study variance in meta-analysis.

The effect sizes of studies included in a meta‐analysis do often not share a common true effect size due to differences in for instance the design of the studies. Estimates of this so‐called between‐study variance are usually imprecise. Hence, reporting a confidence interval together with a point estimate of the amount of between‐study variance facilitates interpretation of the meta‐analytic results. Two methods that are recommended to be used for creating such a confidence interval are the Q‐profile and generalized Q‐statistic method that both make use of the Q‐statistic. These methods are exact if the assumptions underlying the random‐effects model hold, but these assumptions are usually violated in practice such that confidence intervals of the methods are approximate rather than exact confidence intervals. We illustrate by means of two Monte‐Carlo simulation studies with odds ratio as effect size measure that coverage probabilities of both methods can be substantially below the nominal coverage rate in situations that are representative for meta‐analyses in practice. We also show that these too low coverage probabilities are caused by violations of the assumptions of the random‐effects model (ie, normal sampling distributions of the effect size measure and known sampling variances) and are especially prevalent if the sample sizes in the primary studies are small.

Feasible generalized least squares using support vector regression

We investigate semiparametric Feasible Generalized Least Squares using Support Vector Regression to estimate the conditional variance function. Monte Carlo results indicate the resulting estimator and an accompanying standard error correction offer substantially improved precision, nominal coverage rates, and shorter confidence intervals than Ordinary Least Squares with heteroskedasticity-consistent standard errors. Reductions in root mean squared error can be over 90% of those achievable when the form of heteroskedasticity is known.

Adaptive multigroup confidence intervals with constant coverage

SUMMARYCommonly used interval procedures for multigroup data attain their nominal coverage rates across a population of groups on average, but their actual coverage rate for a given group will be above or below the nominal rate, depending on the group mean. While correct coverage for a given group can be achieved with a standard $t$-interval, this approach is not adaptive to the available information about the distribution of group-specific means. In this article we construct confidence intervals that have a constant frequentist coverage rate and that make use of information about across-group heterogeneity, resulting in constant-coverage intervals that are narrower than standard $t$-intervals on average across groups. Such intervals are constructed by inverting biased Bayes-optimal tests for the mean of each group, where the prior distribution for a given group is estimated with data from the other groups.

Guaranteed-Performance Time-Varying Formation Control for Swarm Systems Subjected to Communication Constraints

Guaranteed-performance time-varying formation analysis and design problems for general high-order swarm systems with communication constraints are investigated. First, a guaranteed-performance time-varying formation control protocol is proposed by using the intermittent relative local information, and the performance function is introduced to guarantee the formation regulation performance among neighboring agents. An explicit presentation of the formation center function is presented to describe the movement trajectory of the formation as a whole. Then, sufficient conditions for guaranteed-performance time-varying formation analysis and design are respectively given by linear matrix inequality techniques, and a guaranteed-performance cost is determined meanwhile. When the desired guaranteed-performance time-varying formation satisfies the time-varying formation feasibility condition, swarm systems with communication constraints can achieve the guaranteed-performance time-varying formation by designing the gain matrix of the guaranteed-performance time-varying formation control protocol if the nominal coverage rate is larger than a positive threshold. Finally, a numerical simulation is shown to illustrate theoretical results.

Inference on the ratio of two coefficients of variation of two lognormal distributions

ABSTRACTThe coefficient of variation (CV) can be used as an index of reliability of measurement. The lognormal distribution has been applied to fit data in many fields. We developed approximate interval estimation of the ratio of two coefficients of variation (CsV) for lognormal distributions by using the Wald-type, Fieller-type, log methods, and method of variance estimates recovery (MOVER). The simulation studies show that empirical coverage rates of the methods are satisfactorily close to a nominal coverage rate for medium sample sizes.

Feasible Generalized Least Squares Using Machine Learning

In the presence of heteroskedastic errors, regression using Feasible Generalized Least Squares (FGLS) offers potential efficiency gains over Ordinary Least Squares (OLS). However, FGLS adoption remains limited, in part because the form of heteroskedasticity may be misspecified. We investigate machine learning methods to address this concern, focusing on Support Vector Regression. Monte Carlo results indicate the resulting estimator and an accompanying standard error correction offer substantially improved precision, nominal coverage rates, and shorter confidence intervals than OLS with heteroskedasticity-consistent (HC3) standard errors. Reductions in root mean squared error are over 90% of those achievable when the form of heteroskedasticity is known.

A new prediction-based variance estimator for two-stage model-assisted surveys of forest resources

Forest resource assessments utilizing remotely sensed auxiliary data are becoming increasingly important due to their ability to provide precise estimates of forest parameters at low cost. In presenting results from such surveys, it is important to provide not only estimates of the target parameters, but also their confidence intervals, which provide the range of values wherein the true value is located with a certain level of confidence. If such an interval is narrow the point estimates from the survey can be considered very reliable. In estimating the confidence interval the variance of an estimator must first be estimated. Unbiasedness, i.e. that an estimator on average coincides with the true value, is an important property also for variance estimators. Another important property is that the variance estimator itself has low variance, not least in cases when the variance estimates obtained with the estimator may not be strictly positive. One such important case is when two-stage designs are used to first allocate sample clusters in the form of strips from which auxiliary data, such as metrics derived from airborne laser scanning, are obtained; field data are then derived from sample plots beneath each sample strip in a second stage. In this article we compare two variance estimators for such surveys. The first estimator is a standard estimator suggested in reference textbooks on model-assisted sampling. The second estimator is proposed by the authors, and utilizes the auxiliary data to a larger extent. Through Monte Carlo simulation we show that both variance estimators are approximately unbiased, but that the new estimator is more stable (i.e., has lower empirical variance) and provides empirical confidence interval coverage rates that coincide more closely with the nominal coverage rates.

Confidence intervals for directly standardized rates using mid-p gamma intervals.

Directly standardized rates continue to be an integral tool for presenting rates for diseases that are highly dependent on age, such as cancer. Statistically, these rates are modeled as a weighted sum of Poisson random variables. This is a difficult statistical problem, because there are k observed Poisson variables and k unknown means. The gamma confidence interval has been shown through simulations to have at least nominal coverage in all simulated scenarios, but it can be overly conservative. Previous modifications to that method have closer to nominal coverage on average, but they do not achieve the nominal coverage bound in all situations. Further, those modifications are not central intervals, and the upper coverage error rate can be substantially more than half the nominal error. Here we apply a mid-p modification to the gamma confidence interval. Typical mid-p methods forsake guaranteed coverage to get coverage that is sometimes higher and sometimes lower than the nominal coverage rate, depending on the values of the parameters. The mid-p gamma interval does not have guaranteed coverage in all situations; however, in the (not rare) situations where the gamma method is overly conservative, the mid-p gamma interval often has at least nominal coverage. The mid-p gamma interval is especially appropriate when one wants a central interval, since simulations show that in many situations both the upper and lower coverage error rates are on average less than or equal to half the nominal error rate.

Weighted analysis of composite endpoints with simultaneous inference for flexible weight constraints.

Composite endpoints are widely used as primary endpoints of randomized controlled trials across clinical disciplines. A common critique of the conventional analysis of composite endpoints is that all disease events are weighted equally, whereas their clinical relevance may differ substantially. We address this by introducing a framework for the weighted analysis of composite endpoints and interpretable test statistics, which are applicable to both binary and time‐to‐event data. To cope with the difficulty of selecting an exact set of weights, we propose a method for constructing simultaneous confidence intervals and tests that asymptotically preserve the family‐wise type I error in the strong sense across families of weights satisfying flexible inequality or order constraints based on the theory of χ¯2‐distributions. We show that the method achieves the nominal simultaneous coverage rate with substantial efficiency gains over Scheffé's procedure in a simulation study and apply it to trials in cardiovascular disease and enteric fever. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

SMALL BANDWIDTH ASYMPTOTICS FOR DENSITY-WEIGHTED AVERAGE DERIVATIVES

This paper proposes (apparently) novel standard error formulas for the density-weighted average derivative estimator of Powell, Stock, and Stoker (Econometrica 57, 1989). Asymptotic validity of the standard errors developed in this paper does not require the use of higher-order kernels, and the standard errors are “robust” in the sense that they accommodate (but do not require) bandwidths that are smaller than those for which conventional standard errors are valid. Moreover, the results of a Monte Carlo experiment suggest that the finite sample coverage rates of confidence intervals constructed using the standard errors developed in this papercoincide (approximately) with the nominal coverage rates across a nontrivial range of bandwidths.

Estimating hedged portfolio value-at-risk using the conditional copula: An illustration of model risk

The conventional portfolio value-at-risk model with the assumption of normal joint distribution, which is commonly practiced, exhibits considerable biases due to model specification errors. This paper utilizes the estimation of hedged portfolio value-at-risk (HPVaR) to illustrate the potential model risk due to inappropriate use of the correlation coefficient and normal joint distribution between index spot and futures returns. The results show that HPVaR estimation can be improved by using the conditional copulas and their mixture models to form joint distributions to calculate the optimal hedge ratio. Backtesting diagnostics indicate that the copula-based HPVaR outperforms the conventional HPVaR estimator at both the 99% and the 95% coverage rates. The conventional models obviously underestimate the HPVaR, especially under a 99% coverage rate. We then employ a bootstrap resampling technique to quantify and compare the magnitude of model risk by constructing confidence intervals around HPVaR point estimates. The results suggest that the risk management models should apply a smaller nominal coverage rate (95% instead of 99%) to avoid the model risk mentioned above.