False discovery rate control for non-positively regression dependent test statistics

Abstract

In this paper we present a modification of the Benjamini and Hochberg false discovery rate controlling procedure for testing non-positive dependent test statistics. The new testing procedure makes use of the same series of linearly increasing critical values. Yet, in the new procedure the set of p-values is divided into subsets of positively dependent p-values, and each subset of p-values is separately sorted and compared to the series of critical values. In the first part of the paper we introduce the new testing methodology, discuss the technical issues needed to apply the new approach, and apply it to data from a genetic experiment. In the second part of the paper we discuss pairwise comparisons. We introduce FDR controlling procedures for testing pairwise comparisons. We apply these procedures to an example extensively studied in the statistical literature, and to test pairwise comparisons in gene expression data. We also use the new testing procedure to prove that the Simes procedure can, in some cases, be used to test all pairwise comparisons. The control over the FDR has proven to be a successful alternative to control over the family wise error rate in the analysis of large data sets; the Benjamini and Hochberg procedure has also made the application of the Simes procedure to test the complete null hypothesis unnecessary. Our main message in this paper is that a more conservative approach may be needed for testing non-positively dependent test statistics: apply the Simes procedure to test the complete null hypothesis; if the complete null hypothesis is rejected apply the new testing approach to determine which of the null hypotheses are false. It will probably yield less discoveries, however it ensures control over the FDR.