Two-sample Behrens–Fisher problems for high-dimensional data: A normal reference approach

Abstract

High-dimensional data are frequently encountered with the development of modern data collection techniques. Testing the equality of the mean vectors of two high-dimensional samples with possibly different covariance matrices is usually referred to as a high-dimensional two-sample Behrens–Fisher (BF) problem. In the high-dimensional setting, the classical BF solutions are expected to perform poorly or become inapplicable due to the singularity of the sample covariance matrices. Several approaches have been proposed in the literature to address this challenging issue but they all require strong regularity conditions on the underlying covariance matrices to guarantee that their test statistics are asymptotically normally distributed. To overcome this difficulty, an L2-norm-based test is proposed and studied in this article. It is shown that under some regularity conditions and the null hypothesis, the test statistic and a chi-square-type mixture have the same normal or non-normal limiting distribution. It is then natural to approximate the null distribution of the proposed test using that of the chi-square-type mixture, which is actually obtained from the proposed test statistic when the two high-dimensional samples are normally distributed. The resulting test is then referred to as a normal reference test. The distribution of the chi-square-type mixture can then be well approximated by the Welch–Satterthwaite χ2-approximation with the approximation parameters consistently estimated from the data. The asymptotic power of the proposed test is established. Good performance of the proposed test against several existing competitors is demonstrated via several simulation studies and illustrated by a real data example.