Small sample sizes in clinical trials: a statistician’s perspective

Abstract

Small sample sizes can occur in Phase III clinical trials, either by design because the disease is rare or as a result of early closure due to recruitment failure. In either case there is a need to think differently about the statistical analysis, as the more traditional approaches may be problematic. In the case of a rare disease, there is an opportunity to plan the statistical analysis to account for the expected small numbers of patients; whilst in the failed trial, there may be a need to change the statistical analysis plan in order to maximize the usefulness of the information provided by the unexpected smaller number of patients. Clinicians have to make difficult treatment decisions for their patients on a daily basis and although small sample sizes are not ideal, there are ethical arguments to consider. Patients with rare diseases have the right for treatment decisions to be based on some level of unbiased evidence and in a failed trial it is ethical to analyze the data in such a way that the data can still aid decisions and, thereby, provide some return for the investment made by patients and funders. Traditionally, Phase III trial designs are based on hypothesis testing. Typically, this approach tests the null hypothesis of no treatment effect against the alternative hypothesis that there is a treatment effect. The size of the trial is based on maximizing the chances of making a correct conclusion from the trial data; in particular, trials are designed to have a good chance (usually 90%) of rejecting the null hypothesis (at a 5% significance level) when a prespecified minimum clinically relevant treatment effect truly exists, a feature known as power. The problem with this approach in a trial with small sample size is that the analysis will be underpowered and the trial is unlikely to make the correct conclusion. Less conventional methodological approaches are supported if they help to improve the interpretability of trial results [1,2]. Clinical trials aim to gather unbiased evidence regarding a treatment effect but, rather than trying to provide a definitive answer through hypothesis testing, an alternative view is to consider trials as a way of reducing uncertainty about the size of a treatment effect. If one starts from the premise that there is considerable uncertainty regarding this unknown quantity, then data from even small numbers of patients in a well-designed clinical trial will make steps towards reducing that uncertainty. This improved information will help clinicians in the treatment decisions that they need to make with their patients. This alternative statistical view lends itself to using a Bayesian approach to analysis [3,4]. This was the view and methodology proposed by one of the earliest papers to discuss designing trials in rare diseases [5] and the Bayesian approach was also advocated at that time more generally in relation to small clinical trials [6]. We support this Bayesian approach, but there are issues in its implementation that we would like to highlight in this editorial. 1MRC Midland Hub for Trials Methodology Research, University of Birmingham, Birmingham, UK 2Cancer Research UK Clinical Trials Unit, University of Birmingham, Birmingham, UK 3University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK *Author for correspondence: E-mail: l.j.billingham@bham.ac.uk