Stepwise normal theory multiple test procedures controlling the false discovery rate

Abstract

Abstract The false discovery rate (FDR), or the expected proportion of falsely rejected null hypotheses to rejected null hypotheses, has recently been proposed as an error rate that multiple testing procedures should in certain circumstances control. So far, only a step-up procedure for independent test statistics has been created explicitly to control the FDR ( Benjamini and Hochberg, 1995 ). In this paper, step-down and step-up procedures are described which asymptotically (as N→∞) control the FDR when the test statistics are the t statistics from consistent multivariate normal estimators of the tested parameters. Determination of the necessary critical constants for the normal theory procedures is achieved using numerical integration when the correlations are equal, or through simulation using the multivariate t distribution when the correlations are arbitrary. The critical constants of the normal theory procedures are compared to those of the Benjamini and Hochberg procedure under the normal assumption, and a large potential power increase is found. Simulation strongly supports the use of critical constants, obtained by an asymptotic argument, in small samples for as many as 30 tests. Adjusted FDR values can be found to quantify the evidence against a given hypothesis.