On the term “randomization test”

Abstract

There is no consensus on the meaning of the term “randomization test”. Contradictory uses of the term are leading to confusion, misunderstandings and indeed invalid data analyses. A main source of the confusion is that the term was not explicitly defined when it was first used in the 1930’s. Later authors made clear proposals to reach a consensus regarding the term. This resulted in some level of agreement around the 1970’s. However, in the last few decades, the term has often been used in ways that contradict these proposals. This paper provides an overview of the history of the term per se, for the first time tracing it back to 1937. This will hopefully lead to more agreement on terminology and less confusion on the related fundamental concepts.