Note on the Two-Sample Problem

Abstract

A recent article (Penfield & McSweeney, 1968) discussed the possible error in applying the two-sample t test to situations in which the assumptions of independence, normality, and variance equality were violated. They suggested the use of three normal scores tests and compared their use with that of the nonparametric Mann-Whitney U test. There is, however, an alternative procedure which can be quite useful in generating exact tests of hypotheses under randomization models. This involves the use of permutation tests (Fisher, 1925) which are very simple to define and can be effectively applied in small sample situations. In fact, many researchers are interested in permutation tests because the fixedeffects P test is often a good approximation to the permutation test. In addition, this latter statistic is not based upon the restrictive normal theory model and does provide an exact test of the hypothesis of equality of means. Basically permutation tests (also known as randomization tests) involve considering all possible arrangements of the data from an experiment. For example, consider the data from Exp. 2 of the Penfield and McSweeney article. They were studying the effect of previous experience with analogy items on a student's Miller Analogies Test score. The scores they reported for the PT group (previous training with analogies) and the NT group (no training) are given below. PT: 73, 59, 46, 68, 62 NT: 43, 38, 56