FDR-controlling stepwise procedures and their false negatives rates

Abstract

The concept of false discovery rate (FDR), which is the expected proportion of false positives (type I errors) among rejected hypotheses, has received increasing attention recently by researchers in multiple hypotheses testing. A similar measure involving false negatives (type II errors), which we call the false negatives rate (FNR), is considered and an explicit formula of it is developed for a generalized step-up-step-down procedure in terms of probability distributions of ordered test statistics. It is shown that a step-down procedure can be used to control the FNR under certain conditions on the test statistics. Various FDR-controlling procedures that exist in a given multiple testing situation are further studied in terms of the FNR. In particular, an unbiasedness property is defined for an FDR-controlling multiple testing procedure by the inequality FDR+FNR⩽1. The FDR-controlling generalized step-up-step-down procedure considered in Sarkar (Ann. Statist. 26 (2002) 239), in particular the Benjamini and Hochberg (J. Roy. Statist. Soc. Ser. B 57 (1995) 289) step-up procedure, is proved to be unbiased when the test statistics are independent and is conjectured to be so based on simulations when they are dependent. Also conjectured is the unbiasedness of the FDR-controlling Benjamini and Lui (J. Statist. Plann. Inference 82 (1999) 163) step-down procedure with independent as well as dependent test statistics. Different FDR-controlling procedures are investigated based on simulated data from equi-correlated multivariate normals in terms of the quantity 1−FNR−FDR that reflects the strength of unbiasedness.