The Power and the Pitfalls of Randomized Clinical Trials

Abstract

In this issue of The Annals of Thoracic Surgery, the review article by Gaudino and colleagues1Gaudino M. Ruel M. Obadia J.-F. et al.Methodologic considerations on four cardiovascular interventions trials with contradictory results.Ann Thorac Surg. 2021; 111: 690-700Abstract Full Text Full Text PDF PubMed Scopus (7) Google Scholar beautifully illustrates challenges associated with randomized clinical trials (RCTs) in surgery. The authors demonstrated the impact of differences in outcome definitions and patient populations—and potentially open-label design—on study results, calling for agreement on common outcomes definitions and duration of follow-up in study design and the involvement of statistical methodologists in guideline documents. Here, here! RCTs are widely accepted as the most objective method to evaluate treatment effect of an intervention by neutralizing known and unknown confounders. As such they stand at the top of the “evidence pyramid.” However they suffer significant limitations, particularly when related to interventional and operative procedures. First, bias is inescapable. The RCT design addresses “treatment selection bias” but inescapably substitutes “entry selectivity bias.” Despite complaints that the population studied in a given RCT are not representative of “real-world” clinical practice, such entry bias is necessary if one is to accurately study the treatment effect. This is analogous to using inbred mouse strains in basic science or when evaluating a specific pharmacologic agent for example. Appropriately these authors focus on the need to clarify and emphasize the impact of this bias rather than complain about its existence. Indeed they highlight the potential influence of heterogeneity in study population in the MITR-FR trial on study results. In fact in clever hands subtle manipulation of the study population (and other subtleties in RCT study design) can be used to demonstrate equivalence of empty backpacks to parachutes among individuals jumping from aircraft.2Yeh R.W. Valsdottir L.R. Yeh M.W. et al.Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial.BMJ. 2018; 363: k5094Crossref PubMed Scopus (65) Google Scholar The authors also emphasized the impact of differences in composite endpoints, perhaps notoriously including the definitions of their components, on study outcomes. It should be more broadly noted, however, that composite endpoints in all instances are potentially misleading. Used to increase the power of a study, they are commonly odd bedfellows chosen more as a matter of convenience than equivalence. Although repeat revascularization is indeed undesirable and potentially dangerous, would any of us seriously equate it with stroke or death? Among the challenges in surgical RCTs unaddressed by these authors is heterogeneity in the treatment administered (procedure performed). The consequences of this vulnerability is evident in the Arterial Revascularization Trial study, which demonstrated no impact of bilateral internal thoracic artery grafts when analyzed by intention to treat, but the opposite when considered by treatment received.3Taggart D.P. Benedetto U. Gerry S. et al.Arterial Revascularization Trial Investigators. Bilateral versus single internal thoracic artery grafts at 10 years.N Engl J Med. 2019; 380: 437-446Crossref PubMed Scopus (206) Google Scholar Operations are not drugs, and this impact increases with the complexity of the intervention. We are challenged to participate more enthusiastically in RCTs. I agree that it is incumbent on us to understand the nuances of their design and conduct. I also appreciate their call for statistical methodologists to participate in the construct of guidelines documents. Too often we call for “subject matter experts” to write these documents, which of course only serves to increase the impact of “expert opinion” which, despite our appetite, sits at the bottom of the aforementioned evidence pyramid. Methodologic Considerations on Four Cardiovascular Interventions Trials With Contradictory ResultsThe Annals of Thoracic SurgeryVol. 111Issue 2PreviewContradictory findings from randomized trials addressing similar research questions are not uncommon in medicine. Although differing results may reflect true differences in the treatment effects or in the deliverability of the intervention, more commonly it is as a consequence of small but important discrepancies in study design. Full-Text PDF