Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing

Abstract

Consider the problem of testing k hypotheses simultaneously. In this paper, we discuss finite and large sample theory of stepdown methods that provide control of the familywise error rate (FWE). To improve upon the Bonferroni method or Holm's stepdown method, Westfall and Young (1993) make use of resampling to construct stepdown methods that implicitly estimate the dependence structure of the test statistics. However, their methods depend on the assumption of subset pivotality. Our goal is to construct general stepdown methods that do not require such an assumption. It turns out that a key component for the validity of stepdown methods is a monotonicity requirement of critical values. By imposing such monotonicity on estimated critical values, we show that the problem of constructing a valid multiple test procedure controling the FWE can be reduced to the problem of contructing a single test controling the usual probability of a Type 1 error.