Usual Confidence Research Articles

Confidence intervals in general regression models that utilize uncertain prior information

– We consider a general regression model, either without a scale parameter or with a known scale parameter. We construct a confidence interval for a scalar parameter of interest that utilizes the uncertain prior information that a distinct scalar parameter takes a specified value. This confidence interval has excellent coverage properties. It also has local scaled expected length, where the scaling is with respect to the usual confidence interval, that (a) is substantially less than 1 when the prior information is correct, (b) has a maximum value that is not too large, and (c) is close to 1 when the data and prior information are highly discordant. An important practical application of this confidence interval is to a quantal bioassay carried out to compare two similar compounds. In this context, the uncertain prior information is that the hypothesis of “parallelism” holds. We provide extensive numerical results that illustrate the attractive properties of this confidence interval in this context.

Association rules and decision rules

AbstractDetermining association rules of significant interest is an essential task within data mining and statistical analysis. In this paper, we first precisely define the notion of association rule. For this, we introduce a general model, which includes the usual transaction model, and which allows many operations on the association rules. Then, we interpret association rules as statistical decision rules. This interpretation leads to four decisional measures, one of them being the usual confidence. Then, we give some strategies based on the use of these four decisional measures in order to select or to construct association rules with a given consequent. We finally present an experimental study to illustrate these strategies. This study is carried out in R language, with the R‐package we specifically built for association rules mining.

Confidence intervals that utilize sparsity

We consider a linear regression model with orthogonal regressors and Gaussian random errors with known variance, in the low‐dimensional setting that the length of the regression parameter vector does not exceed the length of the response vector. We suppose that we have uncertain prior information that sparsity holds, that is, that many of the components of the regression parameter vector are zero. Our aim is to construct confidence intervals for the components of this vector with both the desired coverage probability and attractive expected length properties, particularly when sparsity holds. It is known that for fixed‐width confidence intervals centered on penalized maximum likelihood estimators, such as hard‐thresholding, LASSO and SCAD estimators, the achievement of the desired minimum coverage necessarily results in very unattractive expected length properties. We present confidence intervals, with data‐based widths, which achieve our aim. These confidence intervals dominate the usual confidence intervals with the same coverage probability, whenever the degree of sparsity exceeds a known value.

The significance filter, the winner's curse and the need to shrink

The “significance filter” refers to focusing exclusively on statistically significant results. Since frequentist properties such as unbiasedness and coverage are valid only before the data have been observed, there are no guarantees if we condition on significance. In fact, the significance filter leads to overestimation of the magnitude of the parameter, which has been called the “winner's curse.” It can also lead to undercoverage of the confidence interval. Moreover, these problems become more severe if the power is low. These issues clearly deserve our attention. They have been studied mostly through empirical observation and simulation, while there are relatively few mathematical results. Here we study them both from the frequentist and the Bayesian perspective. We prove that the relative bias of the magnitude is a decreasing function of the power and that the usual confidence interval undercovers when the power is less than 50%. We conclude that it is important to apply the appropriate amount of shrinkage to counter the winner's curse.

CONFIDENCE INTERVALS CONSTRUCTED BY MODEL AVERAGING AND BOOTSTRAP SMOOTHING

We assess the performance of confidence intervals constructed by model averaging and bootstrap smoothing, using a simple testbed situation of two nested linear regression models with either known error variance or unknown error variance. The assessment is carried out by deriving computationally convenient exact expressions for the coverage probability and scaled expected length where the scaling is with respect to the usual confidence interval, with the same minimum coverage, based on the full model. We assessed the properties of the confidence interval centered on the model averaged estimator proposed by Buckland et al. (1997) with half-width proportional to the standard error of the estimate put forward by Buckland et al. (1997) of their formula (9). We considered both the finite sample case and the asymptotic case. Our results show that this model averaged confidence interval cannot be generally recommended, although it performs well when the error degrees of freedom is 1. We also assessed the properties of the confidence intervals centered on the bootstrap smoothed estimator with half-width proportional to the estimate of (a) the standard deviation of this estimator and (b) the delta method approximation, derived by Efron (2014), to the standard deviation of this estimator. We show that, except for the case of unknown error variance and small error degrees of freedom, the confidence interval centered on the bootstrap smoothed estimator does not have scaled expected length that is substantially less than 1, when the simpler model is correct. For this reason, we sought a formula for the data-based width that leads to improved performance of the confidence interval centered on the bootstrap smoothed estimator. Since this search failed, it raises the following question. Is there any formula for the data-based width of such a confidence interval so that it performs well? To answer this question, we compute the performance bound due to Kabaila & Kong (2016).

Gestão de carteiras sob múltiplos regimes: Performance fora da amostra

Propomos uma estratégia de alocação dinâmica para um investidor que considera três regimes econômicos latentes, estimados através dos retornos de cinco classes de ativos brasileiros. A estratégia de alocação é definida a partir de uma solução analítica aproximada numa configuração realista, porém a ponto de não permitir solução analítica exata. Testamos a performance da carteira fora da amostra e o resultado superou o retorno de todos os benchmarks analisados em 6 dos 10 anos em tela, tendo apresentado média de retorno semanal superior a qualquer dos benchmarks com significância estatística. De 2009 a 2019, nossa estratégia alcançou um retorno médio igual a 21,6% ao ano contra, por exemplo, 9,8% a.a. do CDI e 4,7% a.a. do Ibovespa. Concluímos que a solução proposta pode interessar a gestores de investimentos e objetivamos fomentar a aplicação de modelos de múltiplos regimes no mercado brasileiro de gestão de carteiras de investimentos.

When is n large enough? Looking for the right sample size to estimate proportions

ABSTRACTThe central limit theorem indicates that when the sample size goes to infinite, the sampling distribution of means tends to follow a normal distribution; it is the basis for the most usual confidence interval and sample size formulas. This study analyzes what sample size is large enough to assume that the distribution of the estimator of a proportion follows a Normal distribution. Also, we propose the use of a correction factor in sample size formulas to ensure a confidence level even when the central limit theorem does not apply for these distributions.

Confidence intervals centred on bootstrap smoothed estimators

SummaryBootstrap smoothed (bagged) parameter estimators have been proposed as an improvement on estimators found after preliminary data‐based model selection. A result of Efron in 2014 is a very convenient and widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. This approximation provides an easily computed guide to the accuracy of this estimator. In addition, Efron considered a confidence interval centred on the bootstrap smoothed estimator, with width proportional to the estimate of this approximation to the standard deviation. We evaluate this confidence interval in the scenario of two nested linear regression models, the full model and a simpler model, and a preliminary test of the null hypothesis that the simpler model is correct. We derive computationally convenient expressions for the ideal bootstrap smoothed estimator and the coverage probability and expected length of this confidence interval. In terms of coverage probability, this confidence interval outperforms the post‐model‐selection confidence interval with the same nominal coverage and based on the same preliminary test. We also compare the performance of the confidence interval centred on the bootstrap smoothed estimator, in terms of expected length, to the usual confidence interval, with the same minimum coverage probability, based on the full model.

On confidence intervals centred on bootstrap smoothed estimators

Bootstrap smoothed (bagged) estimators have been proposed as an improvement on estimators found after preliminary data‐based model selection. Efron derived a widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. He also considered a confidence interval centred on the bootstrap smoothed estimator, with width proportional to the estimate of this standard deviation. Recently, Kabaila and Wijethunga assessed the performance of this confidence interval in the scenario of two nested linear regression models, the full model and the simpler model, for the case of known error variance and preliminary model selection using a hypothesis test. They found that the performance of this confidence interval was not substantially better than the usual confidence interval based on the full model, with the same minimum coverage. We extend this assessment to the case of unknown error variance by deriving a computationally convenient exact formula for the ideal (i.e., in the limit as the number of bootstrap replications diverges to infinity) delta method approximation to the standard deviation of the bootstrap smoothed estimator. Our results show that, unlike the known error variance case, there are circumstances in which this confidence interval has attractive properties.

Wilson Confidence Intervals for Binomial Proportions With Multiple Imputation for Missing Data

ABSTRACTWe present a Wilson interval for binomial proportions for use with multiple imputation for missing data. Using simulation studies, we show that it can have better repeated sampling properties than the usual confidence interval for binomial proportions based on Rubin’s combining rules. Further, in contrast to the usual multiple imputation confidence interval for proportions, the multiple imputation Wilson interval is always bounded by zero and one. Supplementary material is available online.

Counting by weighing: construction of two-sided confidence intervals

ABSTRACTCounting by weighing is widely used in industry and often more efficient than counting manually which is time consuming and prone to human errors especially when the number of items is large. Lower confidence bounds on the numbers of items in infinitely many future bags based on the weights of the bags have been proposed recently in Liu et al. [Counting by weighing: Know your numbers with confidence, J. Roy. Statist. Soc. Ser. C 65(4) (2016), pp. 641–648]. These confidence bounds are constructed using the data from one calibration experiment and for different parameters (or numbers), but have the frequency interpretation similar to a usual confidence set for one parameter only. In this paper, the more challenging problem of constructing two-sided confidence intervals is studied. A simulation-based method for computing the critical constant is proposed. This method is proven to give the required critical constant when the number of simulations goes to infinity, and shown to be easily implemented on an ordinary computer to compute the critical constant accurately and quickly. The methodology is illustrated with a real data example.

Notice of Retraction: Statistical Figures of Merit for the Feature Selective Validation Method

This paper presents point-by-point feature selective validation (FSV) data as a continuous distribution function, rather than in the more usual confidence histogram form, and from that derives the mean, standard deviation, skewness, and kurtosis. The increased information that this offers is shown by presenting again the data from three previous exercises to verify FSV performance against visual assessment but including the standard deviation, where it is demonstrated that more robust conclusions about FSV overall assessment can be provided. The implication of the use of statistical data within FSV for including uncertainty in data comparisons is discussed.

Confidence distributions applied to propagating uncertainty to inference based on estimating the local false discovery rate: A fiducial continuum from confidence sets to empirical Bayes set estimates as the number of comparisons increases

ABSTRACTEmpirical Bayes estimates of the local false discovery rate can reflect uncertainty about the estimated prior by supplementing their Bayesian posterior probabilities with confidence levels as posterior probabilities. This use of coherent fiducial inference with hierarchical models generates set estimators that propagate uncertainty to varying degrees. Some of the set estimates approach estimates from plug-in empirical Bayes methods for high numbers of comparisons and can come close to the usual confidence sets given a sufficiently low number of comparisons.

Optimized recentered confidence spheres for the multivariate normal mean

Casella and Hwang, 1983, JASA, introduced a broad class of recentered confidence spheres for the mean $\boldsymbol{\theta}$ of a multivariate normal distribution with covariance matrix $\sigma^{2}\boldsymbol{I}$, for $\sigma^{2}$ known. Both the center and radius functions of these confidence spheres are flexible functions of the data. For the particular case of confidence spheres centered on the positive-part James-Stein estimator and with radius determined by empirical Bayes considerations, they show numerically that these confidence spheres have the desired minimum coverage probability $1-\alpha$ and dominate the usual confidence sphere in terms of scaled volume. We shift the focus from the scaled volume to the scaled expected volume of the recentered confidence sphere. Since both the coverage probability and the scaled expected volume are functions of the Euclidean norm of $\boldsymbol{\theta}$, it is feasible to optimize the performance of the recentered confidence sphere by numerically computing both the center and radius functions so as to optimize some clearly specified criterion. We suppose that we have uncertain prior information that $\boldsymbol{\theta}=\boldsymbol{0}$. This motivates us to determine the center and radius functions of the confidence sphere by numerical minimization of the scaled expected volume of the confidence sphere at $\boldsymbol{\theta}=\boldsymbol{0}$, subject to the constraints that (a) the coverage probability never falls below $1-\alpha$ and (b) the radius never exceeds the radius of the standard $1-\alpha$ confidence sphere. Our results show that, by focusing on this clearly specified criterion, significant gains in performance (in terms of this criterion) can be achieved. We also present analogous results for the much more difficult case that $\sigma^{2}$ is unknown.

Admissibility of the Usual Confidence Set for the Mean of a Univariate or Bivariate Normal Population: The Unknown Variance Case

Summary In the Gaussian linear regression model (with unknown mean and variance), we show that the standard confidence set for one or two regression coefficients is admissible in the sense of Joshi. This solves a long-standing open problem in mathematical statistics, and this has important implications on the performance of modern inference procedures post model selection or post shrinkage, particularly in situations where the number of parameters is larger than the sample size. As a technical contribution of independent interest, we introduce a new class of conjugate priors for the Gaussian location–scale model.

Counting by Weighing: Know Your Numbers with Confidence

SummaryCounting by weighing is often more efficient than counting manually, which is time consuming and prone to human errors, especially when the number of items (e.g. plant seeds, printed labels or coins) is large. Papers in the statistical literature have focused on how to count, by weighing, a random number of items that is close to a prespecified number in some sense. The paper considers the new problem, arising from a consultation with a company, of making inference about the number of 1p coins in a bag with known weight for infinitely many bags, by using the estimated distribution of coin weight from one calibration data set only. Specifically, a lower confidence bound has been constructed on the number of 1p coins for each of infinitely many future bags of 1p coins, as required by the company. As the same calibration data set is used repeatedly in the construction of all these lower confidence bounds, the interpretation of coverage frequency of the lower confidence bounds that is proposed is different from that of a usual confidence set.

Skewness-kurtosis adjusted confidence estimators and significance tests

First and second kind modifications of usual confidence intervals for estimating the expectation and of usual local alternative parameter choices are introduced in a way such that the asymptotic behavior of the true non-covering probabilities and the covering probabilities under the modified local non-true parameter assumption can be asymptotically exactly controlled. The orders of convergence to zero of both types of probabilities are assumed to be suitably bounded below according to an Osipov-type condition and the sample distribution is assumed to satisfy a corresponding tail condition due to Linnik. Analogous considerations are presented for the power function when testing a hypothesis concerning the expectation both under the assumption of a true hypothesis as well as under a modified local alternative. A limit theorem for large deviations by S.V. Nagajev/V.V. Petrov applies to prove the results. Applications are given for exponential families.

Proportional Hazard Inverse Weibull Distribution and Associated Inference

In this study, we have considered the proportional hazard version of the inverse Weibull distribution. It has been shown that the density and hazard rate functions are unimodal and the mean residual life function is bathtub shaped. For the considered model, many functions of the parameters, such as the mean, variance, coefficient of variation and the critical points of the density, hazard rate and mean residual life functions cannot be given in explicit form. As a result, the variances of the maximum likelihood estimators of such functions cannot be given in explicit form to construct the usual asymptotic confidence interval. In this study, we use the percentile bootstrap estimation method to estimate such variances to construct the asymptotic confidence intervals. The asymptotic confidence intervals are compared with the bootstrap confidence intervals. Simulation studies are carried out to examine the performance of the maximum likelihood and bootstrap estimators. An example is provided to illustrate the procedure.

On Econometric Inference and Multiple Use of the Same Data

In economics and finance, test statistics and confidence regions are typically probabilistically dependent on previously-used data. However, the Bayesian and classical justifications for usual tests and confidence regions are known to be invalid in this case. Given the success of empirical econometrics, this leads to the question: Why has econometric inference been possible? The paper provides a simple, yet general, answer to the question. We outline a novel econometric theory that i) is immune to conditioning on previously-used data, and that ii) nests model calibration, and most of econometric practices, whether they are labelled Bayesian or classical.

New bootstrap inference for spurious regression problems

Phillips [11] provides asymptotic theory for regressions that relate nonstationary time series including those integrated of order 1, . A practical implication of the literature on spurious regression is that one cannot trust the usual confidence intervals (CIs). In the absence of prior knowledge that two series are cointegrated, it is therefore recommended that we abandon the specification in levels and work with differenced or detrended series. For situations when the specification in levels is sacrosanct we propose new CIs based on the Maximum Entropy bootstrap explained in Vinod and López-de-Lacalle (Maximum entropy bootstrap for time series: The meboot R package, J. Statist. Softw. 29 (2009), pp. 1–19). An extensive Monte Carlo simulation shows that our proposal can provide more reliable conservative CIs than traditional and block bootstrap intervals.