Rationality and the generalization of randomized controlled trial evidence

Abstract

The randomized controlled trial (RCT) is the gold standard of evidence because it has the highest internal validity; that is, its design is better than any other at preventing various sources of systematic bias from confounding our judgement. The preceding assertion has been amply rehearsed. It is precisely what is meant by the RCT’s placement above all other population study designs in the hierarchy of evidence for prevention and treatment decisions. The hierarchy has been the target of a battery of criticisms over the past two decades [1], yet it lives on in the most recent edition of the Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice [2], and the randomized trial still rules. Recurrent among critiques of the RCT are concerns over its external validity [3–13]. The common thread to these arguments is that trials map poorly onto the reality of clinical medicine; the trial participants are not representative of patients routinely encountered by clinicians, and the ideal and pristine trial conditions are too dissimilar from the concrete and messy world of clinical practice. Concerns about the representativeness of trial populations are neither exaggerated nor trivial. A systematic review of 283 RCTs published in major medical journals found that 81.3% of trials excluded patients due to common co-morbidities, 38.5% due to older age and 54.1% due to concurrent use of commonly prescribed