Randomization tests: A new gold standard?

Abstract

Classical statistical methods rely on the analytical power of mathematics and some assumptions rather than on computer power. In research with human participants the assumption of random sampling is rarely correct. The great increase in computer power in recent decades makes available an approach to statistical inference which does not require random sampling, namely randomization tests. For these tests we do need random assignment of conditions or treatments to participants or observation occasions, but this is usually necessary anyway to ensure internal validity. External validity is achieved by replication and nonstatistical reasoning whether we use classical or randomization tests. Small-n and single case investigations (including phase designs) benefit from randomization designs and tests, but they can equally well be used for large-n studies. Software is becoming available for analysis of randomization tests, course materials are being developed, and we may be about to see them become a very common if not the principal statistical technique in our toolbox.