Multivariate and multiple permutation tests

Abstract

In this article, we consider the use of permutation tests for comparing multivariate parameters from two populations. First, the underlying properties of permutation tests when comparing parameter vectors from two distributions P and Q are developed. Although an exact level α test can be constructed by a permutation test when the fundamental assumption of identical underlying distributions holds, permutation tests have often been misused. Indeed, permutation tests have frequently been applied in cases where the underlying distributions need not be identical under the null hypothesis. In such cases, permutation tests fail to control the Type 1 error, even asymptotically. However, we provide valid procedures in the sense that even when the assumption of identical distributions fails, one can establish the asymptotic validity of permutation tests in general while retaining the exactness property when all the observations are i.i.d. In the multivariate testing problem for testing the global null hypothesis of equality of parameter vectors, a modified Hotelling’s T2-statistic as well as tests based on the maximum of studentized absolute differences are considered. In the latter case, a bootstrap prepivoting test statistic is constructed, which leads to a bootstrapping after permuting algorithm. Then, these tests are applied as a basis for testing multiple hypotheses simultaneously by invoking the closure method to control the Familywise Error Rate. Lastly, Monte Carlo simulation studies and an empirical example are presented.