Bonferroni Procedure Research Articles

Is the Simes improved Bonferroni procedure conservative?

Simes (1986) proposed a modified Bonferroni procedure for conducting multiple tests of significance. He proved that, when the n test statistics are independent, his procedure has exact size α. Already Hommel(1983) had considered a similar procedure and shown that it need not be conservative. He gave an attainable upper bound for its anticonservativeness. Here we show that, for one-sided tests based on positively correlated bivariate normal test statistics, the procedure is conservative, and for negatively correlated such variables it is anticonservative. Two-sided tests based on bivariate normal variables are always conservative. Some numerical values are given, which show that for all practical purposes the nominal α-value can be used.

Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods.

Public health researchers are sometimes required to make adjustments for multiple testing in reporting their results, which reduces the apparent significance of effects and thus reduces statistical power. The Bonferroni procedure is the most widely recommended way of doing this, but another procedure, that of Holm, is uniformly better. Researchers may have neglected Holm's procedure because it has been framed in terms of hypothesis test rejection rather than in terms of P values. An adjustment to P values based on Holm's method is presented in order to promote the method's use in public health research.

Consistent Specification Testing Via Nonparametric Series Regression

This paper proposes two consistent one-sided specification tests for parametric regression models, one based on the sample covariance between the residual from the parametric model and the discrepancy between the parametric and nonparametric fitted values ; the other based on the difference in sums of squared residuals between the parametric and nonparametric models. We estimate the nonparametric model by series regression. The new test statistics converge in distribution to a unit normal under correct specification and grow to infinity faster than the parametric rate (n -1/2 ) under misspecification, while avoiding weighting, sample splitting, and non-nested testing procedures used elsewhere in the literature. Asymptotically, our tests can be viewed as a test of the joint hypothesis that the true parameters of a series regression model are zero, where the dependent variable is the residual from the parametric model, and the series terms are functions of the explanatory variables, chosen so as to support nonparametric estimation of a conditional expectation. We specifically consider Fourier series and regression splines, and present a Monte Carlo study of the finite sample performance of the new tests in comparison to consistent tests of Bierens (1990), Eubank and Spiegelman (1990), Jayasuriya (1990), Wooldridge (1992), and Yatchew (1992) ; the results show the new tests have good power, performing quite well in some situations. We suggest a joint Bonferroni procedure that combines a new test with those of Bierens and Wooldridge to capture the best features of the three approaches.

Power and Type I Error Rates of New Pairwise Multiple Comparison Procedures Under Heterogeneous Variances

Abstract The Type I error rates and statistical power of 9 selected multiple comparison procedures were compared in a Monte Carlo study. Data were generated for 3-, 4-, and 5-group ANOVA models, from simulated populations with both homogeneous and heterogeneous variances. A variety of patterns of population means were examined, including completely null, partial-null, and multiple-null patterns. None of the procedures was robust to violations of the assumption of variance homogeneity at nominal alpha levels lower than .10, even with equal sample sizes. The Dunn procedure and modified Bonferroni procedures showed better robustness properties than the Tukey procedure and recent modifications of it did. Power comparisons, conducted at a nominal alpha level of .10, showed the Peritz, Ryan, and Fisher-Hayter tests to be consistently most powerful across the variance conditions examined. However, power differences among these three procedures were minimal.

A Dutch population study of the STR Loci HUMTHO1, HUMFES/FPS, HUMVWA31/1 and HUMF13A1, conducted for forensic purposes.

We report on a Dutch population study of the STR loci HUMTHO1, HUMFES/FPS, HUMVWA31/1, and HUMF13A1, in which we used multiplex amplification and automated fragment detection. Genotype and allele frequencies showed no deviation from Hardy-Weinberg and linkage equilibrium. The improved Bonferroni procedure was used to combine the results of several tests. The power of discrimination of a complete profile exceeded 0.9998. We compared the allele frequencies in the Dutch sample to the frequencies in other populations using a bipilot to visualize alleles and populations simultaneously. The Dutch sample was similar to most other Caucasian samples. The data demonstrate that the genetic systems in this report are a valuable tool for forensic identity testing in The Netherlands.

Genetic structure and variability in Pinusstrobus in Quebec

The level of genetic diversity of natural populations of eastern white pine (Pinusstrobus L.) from Quebec was estimated from allozyme variants of 18 loci coding 12 enzyme systems. On average, a white pine population was polymorphic at 50.6% of loci, had 1.96 alleles and 1.22 effective alleles per locus, and observed and expected heterozygosities of 0.176 and 0.180, respectively. The level of genetic diversity was lower in the populations of the St. Lawrence lowlands than in those of western Quebec. This observation will help in guiding the selection program of the eastern white pine improvement program under way in Quebec. Genetic differentiation among sampled populations was weak and accounted for only 2% of the total diversity. The estimate of gene flow was very high, resulting in low values for genetic distances among populations. Only one locus showed a heterogeneity of allelic frequencies among populations after the Bonferroni procedure was applied for simultaneous statistical tests. A cluster analysis based on genetic distances among populations revealed that the Anticosti and Abitibi populations, located at the limit of the natural range of white pine, were similar to populations from regions that were geographically the most distant.

Construction of a Conservative Confidence Region from Projections of an Exact Confidence Region in Multiple Linear Regression

Abstract The problem of constructing a confidence region for simultaneously estimating p, p ≥ 2, linear regression parameters for which confidence statements can be made on the individual parameters is revisited. Here, an intercept may be included among the p paremeters. The technique is due to Working and Hotelling (1929) and Scheffe (1959) and uses the p separate projections of the exact (1 – α) 100% confidence ellipsoid (ellipse if p = 2) to give confidence intervals for each regression parameter. The Cartesian product of these p confidence intervals gives a p-dimensional rectangle that contains the confidence ellipsoid and hence has a joint confidence coefficient of at least (1–α). A simple calculus proof is given to determine these projections. The projection procedure is compared with the Bonferroni procedure for this case.

A comparison of the Bonferroni and Scheffé bounds

Abstract For the general linear model there are many ways to obtain simultaneous confidence intervals. Among them are the Bonferroni, Scheffe and several Sidak-type procedures. The comparisons of these procedures have been extensively studied. However, previously the comparison between the Bonferroni and Scheffe procedures has only been based on numerical inspection or on a comparison with infinite error degrees of freedom. In this paper, we show surprisingly that for one or two degrees of freedom for error. Scheffe's procedure is better, that is, produces shorter intervals, than the Bonferroni's procedure. For more than two degrees of freedom for error. Bonferroni is better than Scheffe for standard levels of confidence, but the reverse can be true for nonstandard confidence levels.

Using Planned Comparisons in Management Research: A Case for the Bonferroni Procedure

This article describes the Bonferroni multiple-comparison procedure, and makes a case for researchers’ more frequent and appropriate use of it. The procedure is discussed as a test that facilitates investigation of precise and powerful a priori multiple comparisons. Characteristics of the Bonferroni procedure are described in relation to the more familiar Scheffe post hoc multiple-comparison method, and a step-by-step guide for comparing and choosing between the two is provided. The Bonferroni procedure is discussed in detail in the context of one-factor analysis-of-variance designs. Application of the technique is then considered in the context of factorial designs, analyses of covariance, univariate repeated-measures analyses, multivariate analyses of variance, and recent sequential hypothesis-testing extensions. To aid the presentation, an example from the field of management is included.

Using planned comparisons in management research: A case for the bonferroni procedure

This article describes the Bonferroni multiple-comparison procedure, and makes a case for researchers' more frequent and appropriate use of it. The procedure is discussed as a test that facilitates investigation of precise and powerful a priori multiple comparisons. Characteristics of the Bonferroni procedure are described in relation to the more familiar Scheffé post hoc multiple-comparison method, and a step-by-step guide for comparing and choosing between the two is provided. The Bonferroni procedure is discussed in detail in the context of one-factor analysis-of-variance designs. Application of the technique is then considered in the context of factorial designs, analyses of covariance, univariate repeated-measures analyses, multivariate analyses of variance, and recent sequential hypothesis-testing extensions. To aid the presentation, an example from the field of management is included.

Adjusted P-Values for Simultaneous Inference

This paper proposes that results from simultaneous tests be reported as adjusted P-values such that, if the adjusted P-value for an individual hypothesis is less than the chosen significance level of a, then the hypothesis is rejected with an experimentwise error rate of no more than ax. Examples are given of adjusted P-values for multiple comparisons in the analysis of variance and of adjusted P-values based on the Bonferroni procedure and modifications of that procedure by Holm (1979, Scandinavian Journal of Statistics 6, 65-70), Hochberg (1988, Biometrika 75, 800-802), and Hommel (1988, Biometrika 75, 383-386). The modified Bonferroni-based procedures are much more powerful than the original Bonferroni procedure, and they deserve wider use. In addition to the above, a procedure is outlined for obtaining adjusted P-values for any closed test procedure.

Morphometric Distinction of Sclerosing Adenosis from Tubular Carcinoma of the Breast

Objective features have been identified that assist in distinguishing sclerosing adenosis from tubular carcinoma of the breast. Hematoxylin and eosin stained paraffin sections of 18 sclerosing adenoses and 18 tubular carcinomas were studied using a TAS Plus image analysis system. Histological measurements from lumens and glands included stereologic features of architecture and morphometry of size and shape (the latter by Fourier coefficients). Cytological measurements included nuclear area, perimeter, diameter and formfactor. Initial analysis suggested utility for several individual features. However, after a modified Bonferroni procedure only two of the features were statistically significant, i.e. the coefficient of variation of luminal form factor and the surface density of glands. Multivariate discriminant analysis using these two variables correctly classified 86% of the cases, with three adenoses and two carcinomas misclassified. Validity of the discriminant rules was supported by classification using measurements from a separate, independent set of cases (ten sclerosing adenoses and nine tubular carcinomas). The classification function computed from the first set misclassified only one case from the second set, a tubular carcinoma, leaving 95% of the cases successfully categorized. Chi square test for 2 x 2 contingency tables gave a p-value < 0.001 for both sets of cases. The results suggest that morphometric features are helpful in distinguishing tubular carcinoma from sclerosing adenosis and point out the need for conservative analysis of high-dimensional data sets.

Community Pharmacy Practice and an Aging Population

Objective: To determine pharmacists' perceptions of the influence of older patients on community pharmacy practice. Data Sources: A mailing list obtained from the South Carolina Board of Pharmacy provided names of community pharmacists by practice setting and gender. Design: A questionnaire was mailed in July 1990 to a random sample of community pharmacists in South Carolina. The questionnaire contained 41 Likert-type opinion statements. Demographic questions relating to practice type and location, pharmacists' educational experiences, and position were included. Data Synthesis: The opinion statements were examined by grouping respondents on the basis of demographic information. Analysis of variance or Student's t-test was used to look for differences among survey responses among various groups of pharmacists. Scheffe's test was used to compare means when the groups were significantly different. The a priori level of significance was 0.05. Critical values were adjusted by the number of statements considered to preserve the error rate at five percent (Bonferroni procedure). Conclusions: The strongest indicator of the impact of the elderly population on community pharmacy is pharmacists' perceived need for more continuing education in geriatrics and gerontology. Economic problems, physician overprescribing, and patient compliance were ranked as the three most difficult aspects of geriatric pharmacy. This study reinforces the need to incorporate geriatric/gerontology education into every pharmacy curriculum.

Step‐down multiple tests for comparing treatments with a control in unbalanced one‐way layouts

We show how a well-known multiple step-down significance testing procedure for comparing treatments with a control in balanced one-way layouts can be applied in unbalanced layouts (unequal sample sizes for the treatments). The method we describe has the advantage that it provides p-values, for each treatment versus control comparison, that take account of the multiple step-down testing nature of the procedure. These joint p-values can be used with any value of alpha, the fixed type I family wise error rate bound, that may be specified by the investigator. To determine the p-values, it is necessary to compute a multivariate Student t integral, for which a computer program is available. This procedure is more powerful than the step-down Bonferroni procedure of Holm and the single-step procedure of Dunnett. An example from the pharmaceutical literature is used to illustrate the procedure.

Efficacy of Caffeine Versus Expectancy in Altering Caffeine-Related Symptoms

The study investigated the independent and interactive effects of caffeine and expectancy on caffeine-related symptoms. High- and low-caffeine consumers were randomly assigned to either an expectancy or nonexpectancy instructional set and one of four caffeine doses. Subjects were administered the State-Trait Anxiety Inventory, (Spielberger & Gorsuch, 1970) and a Symptom Questionnaire (Christensen, White, Krietsch, & Steele, 1990) prior to and 45 min following consumption of one of the four caffeine doses. An analysis of covariance identified a significant main effect for the State-Trait Anxiety Inventory scores and significant main and interaction effects for four Symptom Questionnaire items. However, when the alpha levels were corrected for the increased probability of Type I error, using the Bonferroni procedure, these effects failed to achieve significance. These results suggest that previous reports of subjective caffeine effects are also suspect because of their failure to control for the increased probability of Type I error.

A Modified Bonferroni Method for Discrete Data

The Bonferroni adjustment for multiple comparisons is a simple and useful method of controlling the overall false positive error rate when several significance tests are performed in the evaluation of an experiment. In situations with categorical data, the test statistics have discrete distributions. The discreteness of the null distributions can be exploited to reduce the number of significance tests taken into account in the Bonferroni procedure. This reduction is accomplished by using only the information contained in the marginal totals.

A sequentially rejective test procedure based on a modified Bonferroni inequality

SUMMARY A sharper Bonferroni procedure for multiple tests of significance is derived. This procedure is an improvement of Hochberg's (1988) procedure which contrasts the individual P-values with corresponding critical points. It is shown that Hochberg's original procedure is conservative, and can be made more powerful by enlarging the rejection region so that the type-one error is exactly at the nominal level. It is also shown that the modified procedure retains all the desired properties of the original procedure.

A Comparison of Two Modified Bonferroni Procedures

A Comparison of Two Modified Bonferroni Procedures

Controlling Multiple F Test Errors with an Overall F Test

This article presents a theoretical extension of the problem associated with multiple comparisons among means-the increasing rate of false rejection errors-to multiple F tests of effects in multifactor ANOVAs and regression analyses. The authors empirically confirmed the prediction of such errors through an experiment in which 32% of 100 random computer-generated three-factor ANOVAs had one or more false rejections for the usual seven F tests for main and interaction effects (30% were expected), yet only 6% of the overall F ratios-which tested all effects simultaneously-were falsely declared significant (5% were expected). In contrast, the Bonferroni procedure produced false rejections in 11% of the ANOVAs. The authors conclude that researchers can control the predictable rates of false rejection of errors inherent in multifactor analyses by extending the Fisher protection procedure and requiring the overall F to be significant. The article concludes with hypothetical experiments illustrating how researchers can use the procedure.

Evaluating the cumulative distribution function of a bonferroni test statistic from a type iii bivariate t distribution

While many hypotheses for the bivariate linear model cannot be formulated as the general linear multivariate hypothesis, a Bonferroni procedure using two correlated t statistics is always applicable. The test statistic for this procedure, tmax, is compared with critical values from the univariate t distribution. The test is slightly conservative: it is demonstrated that in practical examples the actual and nominal Type I error rates are close. Percentiles of the tmax, distributions are needed for computing power, error rate, or exact test critical points. The distribution function of tmax, can be evaluated as an integral over the Type III bivariate t distribution of the individual t statistics. A power series algorithm for computing the distribution function of tmax is presented and compared with Monte Carlo algorithms.