A test for the Behrens–Fisher problem based on the method of variance estimates recovery

Abstract

A practical solution based on the method of variance estimates recovery is proposed for the Behrens–Fisher problem. Unlike the classical solution, the proposed test is not based on the central limit theorem and, therefore, it can control Type I error well for small or moderate sample sizes. In this sense, it performs better than the most commonly used Welch's approximate t-test, which may be liberal for small or moderate sample sizes. On the other hand, the powers of the proposed test appears to be close to that of the Welch's approximate t-test and better than those of the Fisher’s solution and the generalized p-value solution; the latter two have no fixed p-value and need to simulate. The proposed test has a meaningful interpretation within and not just across experiments. It can be a competitive alternative solution.