Estimating sample sizes for evidentialttests

Abstract

The evidential approach uses likelihood ratios, typically the natural logarithm of the likelihood ratio representing the amount of evidence for one hypothesis versus another. One of the barriers to using the approach is the unavailability of sample size calculations for commonly used statistical tests. The t test is the most common statistical test used in scientific publications. This paper derives the equations necessary to calculate evidential probabilities and hence the required sample size for different types of t tests. Compared with the conventional Neyman-Pearson approach, the evidential approach requires larger sample sizes. This drawback is countered by the fact that users know the probability for obtaining misleading evidence (strong evidence that points to the wrong hypothesis). Even with small sample sizes, this is quite small (around 0.05) and decreases further with increasing sample sizes. The main challenge faced by the evidential researcher is of obtaining sufficiently strong evidence for or against one of the two specified hypotheses. Like the probability of a Type II error, this probability is large with a small sample size and decreases as the sample size increases. Sample size is estimated by achieving a low probability (e.g. <.1) for the combined probability of misleading and weak evidence.