Multiplicity in oncology randomised controlled trials: a threat to medical evidence?

Abstract

Randomised controlled trials offer several advantages over observational studies in establishing whether cancer treatments improve patient outcomes. First, randomised controlled trials minimise bias and confounding. Observational studies judge treatments that were chosen not at random but deliberately by physicians, and the decision to treat or not treat an individual might be affected by patient characteristics, a type of bias called confounding by indication. Randomised controlled trials prescribe therapy and avoid this limitation. Second, randomised controlled trials restrict multiple hypothesis testing. Randomised controlled trials typically address a specific research question with a prespecified primary outcome and a predefined statistical analysis plan. This approach reduces the ability of investigators to assess multiple outcomes, which in turn minimises the risk of a spurious (eg, false positive) finding. For these reasons, David Sackett placed randomised controlled trials at the apex of the hierarchy of evidence. 1 Guyatt GH Sackett DL Sinclair JC Hayward R Cook DJ Cook RJ Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA. 1995; 274: 1800-1804 Crossref PubMed Scopus (739) Google Scholar Yet in the past 10 years, one of these assumptions is increasingly being questioned. Randomised controlled trials have proliferated in number, often test similar compounds with similar molecular targets, and are often run in redundant and duplicative trial portfolios or agendas. For this reason, multiplicity has emerged as a new threat to the validity of conclusions drawn from randomised controlled trials. In this Comment, we explore this issue. Statistical significance and clinical evidence – Authors' replyWe thank Simon Gates for his reply. In our Comment,1 we highlight an emerging challenge in cancer trials; large portfolios of trials that test a single drug (or very similar drugs) in many different tumour types, even in some settings with limited clinical rationale. By chance alone, some will be positive. This issue must be accounted for. Full-Text PDF Statistical significance and clinical evidenceIn their Comment,1 Vinay Prasad and Christopher M Booth raised an important issue about the interpretation of trial results. However, their proposal of applying Bonferroni-type corrections to p values is likely to be seriously misleading and lead to non-sensical conclusions. For example, if we run a trial and find a result with a p-value of 0·03, and that trial is replicated by another group who get the same result, a reasonable conclusion would be that support for a moderately significant effect is enhanced. Full-Text PDF