Significance tests with restricted randomization design

Abstract

In an experiment comparing the effectiveness of two treatments, A and B, subjects arrive at an experimental site sequentially and must be assigned immediately to one of the two treatment groups. The complete randomization scheme, tossing a fair coin each time independently to decide which treatment will be used for the next subject, has been widely employed because this scheme is free of different kinds of experimental bias (Efron, 1971) and it also provides a basis for statistical inference (Lehmann, 1975, Chapter 1). But in a small experiment complete randomization may result in a severe imbalance between the numbers of subjects assigned to the different treatment groups. A simple way to attain a perfectly balanced experiment is to use the systematic design ABAB.... However, this plan can easily bias the results of the experiment in a number of ways. For example, if the experimenter is predisposed toward treatment A and if he knows or guesses that treatment A will be the next assignment, then he may tend to choose a subject with a high expected response. Because of this bias, at the end of the experiment treatment A might appear to be significantly better than treatment B even if there is in fact no difference between the two. Obviously, the systematic design maximizes this kind of bias. Several compromises between a perfectly balanced assignment scheme and a completely randomized one have been proposed, for example, the permuted block design, the biased coin design (Efron, 1971), the urn design (Wei, 1977, 1978a), the adaptive biased coin design (Wei, 1978b) and the random allocation design (Stigler, 1969; Wei, 1978c). Simon (1979) has given an excellent review. In particular, the urn and adaptive biased coin designs force a small experiment to be balanced, but they tend to the complete randomization scheme as the size of the experiment increases. Recently, a specific urn design has been shown to be D-optimal by Atkinson (1982). Sometimes it may be inappropriate to postulate a population from which the subjects in the experiment have been obtained by random sampling. Instead a randomization model (Lehmann, 1975, Chapter 1) is utilized when the data are analysed. With this approach, the significance tests used for testing the null hypothesis Ho that there is no difference between A and B must be only those tests generated by the experimental randomization design actually employed. For any given sequence of responses from the subjects, one can tabulate all possible patterns of treatment assignments to subjects using the restricted randomization rule and calculate the corresponding treatment