A simple data-driven fallback procedure for multiple comparisons

Abstract

We propose a data-driven fallback procedure to combine the positive aspects of both data-driven multiple comparison procedures and those relying on predetermined strategies. The proposed procedure tests the hypotheses based on the ordering of p-values, but the significance level of the test at each sequential step is accumulated in the manner of the fallback procedure. It is proven that the new procedure strongly controls the familywise error rate and is uniformly more powerful than the weighted Holm procedure for more than two hypotheses. The simulation study shows that the new procedure is more powerful than the fallback in most cases.