Optimally weighted, fixed sequence and gatekeeper multiple testing procedures

Abstract

Abstract We consider a class of closed multiple test procedures indexed by a fixed weight vector. The class includes the Holm weighted step-down procedure, the closed method using the weighted Fisher combination test, and the closed method using the weighted version of Simes’ test. We show how to choose weights to maximize average power, where “average power” is itself weighted by importance assigned to the various hypotheses. Numerical computations suggest that the optimal weights for the multiple test procedures tend to certain asymptotic configurations. These configurations offer numerical justification for intuitive multiple comparisons methods, such as downweighting variables found insignificant in preliminary studies, giving primary variables more emphasis, gatekeeping test strategies, pre-determined multiple testing sequences, and pre-determined sequences of families of tests. We establish that such methods fall within the envelope of weighted closed testing procedures, thus providing a unified view of fixed sequences, fixed sequences of families, and gatekeepers within the closed testing paradigm. We also establish that the limiting cases control the familywise error rate (or FWE), using well-known results about closed tests, along with the dominated convergence theorem.