On dependence assumption in p-value based multiple test procedures

Abstract

ABSTRACT There are various multiple comparison procedures used in confirmatory clinical studies and exploratory research for multiplicity adjustment. Among them are the Hochberg and Benjamini-Hochberg procedures. A common misconception is that these procedures control the type I error rate properly if the test statistics are independent or positively correlated. In fact, a much stronger positive dependence assumption needs to be satisfied to guarantee the type I error rate control. We give a comprehensive review of the dependence conditions used in multiple testing procedures. We show that a weaker positive dependence assumption may result an inflation of type I error rate by a factor of 2 and discuss the type I error rate control under certain negative dependence conditions.