Small Vaccination Experiments with Binary Outcome: The Paradox of Increasing Power with Decreasing Sample Size and/or Increasing Imbalance

Abstract

In animal vaccination experiments with binary outcome (diseased/non diseased), the comparison of the vaccinated and control group is often based on the Fisher exact test. A tool for the evaluation of different designs is proposed, based on the expected power of the Fisher exact test. The expected power can sometimes unexpectedly increase with decreasing sample size and/or increasing imbalance. The reasons for these peculiar results are explained and compared to the results of two other types of tests: the unconditional test and the randomisation test. In a vaccination experiment with a restricted number of animals it is shown to be important to consider expected power in order to choose the most appropriate design.