Evaluating FDR and stratified FDR control approaches for high-throughput biological studies

Abstract

False discovery rate (FDR) control procedures are commonly used for the correction of multiple testing in high-throughput biological studies. Although the expectation of FDR estimations can be controlled, the variance of the FDR estimations has not been fully analysed. Especially, the effect of the variance of the FDR estimator on the stratified FDR control approach, which is proposed to improve the statistical powers of FDR control procedures, is unclear. In this study, we analyzed the effects of three major factors (the percentage of true null hypotheses, the number of hypotheses and the effect size of true alternative hypotheses) on the performances of the FDR and stratified FDR control approaches. We show that the variance of the FDR estimations tends to be small when at least one of the following conditions is satisfied: (1) the percentage of true null hypotheses is not too large, (2) the number of tests is relatively large, or (3) the effect size of true alternative hypotheses is not too small. We demonstrated that when all the hypotheses are stratified into two groups, the variance of the stratified FDR estimations tends to be small if each group satisfies at least one of the above mentioned conditions. In such a situation, the actual stratified FDR for an experiment tends to be under the given control level.