The Future of Clinical Trials Methodology: Accomplishments and Challenges Ahead.

Abstract

See Article, p 668 Clinical trials are essential for the advancement of the practice of medicine. The World Health Organization (WHO) International Standards for Clinical Trial Registries defines an interventional clinical trial as “any research study that prospectively assigns human participants or groups of humans to one or more health-related interventions to evaluate the effects on health outcomes.”1 Clinical trials have evolved enormously since what is thought to be the first written descriptions of a clinical trial in the Old Testament where King Nebuchadnezzar found that people whom he had ordered to consume only meat and wine were less well nourished than those he allowed to have a diet of legumes and water.2 In this issue of Anesthesia & Analgesia, Professor Paul Myles provides an excellent description of the current status and possible future path for clinical trials in anesthesiology and perioperative medicine.3 He correctly identifies the need for improving efficiency, broadening generalizability, and including standardized patient-centered outcomes in trial designs. Although the coronavirus disease 2019 (COVID-19) pandemic has accelerated the acquisition of new data from platform trials such as the Randomized Evaluation of COVID-19 Therapy (RECOVERY) trial with novel adaptive designs, the pandemic has also resulted in substantial collateral damage to other aspects of clinical research. It has been estimated that initiation of and enrollment into non-COVID-19 clinical trials fell by half and those that continued were marked by a large number of protocol deviations and losses to follow-up.4,5 In addition, some patients are now less likely to enroll in a clinical trial than before the pandemic.6 The additional financial burden and shifting clinical priorities related to COVID-19 create further challenges to non-COVID-19 clinical trial research, although some of the benefits, such as innovative models for consent and data collection and accelerated processes for regulatory/ethical approvals, will hopefully persist well beyond the pandemic. Nonetheless, significant progress has been made in the amount of anesthesiology-led clinical trial research, even before COVID-19, as evidenced by a systematic search of randomized trials in the 3 highest impact general medical journals (New England Journal of Medicine [NEJM], Lancet, and Journal of the American Medical Association [JAMA]) from January 1, 2001 to November 30, 2021, in which either of the first 3 or last 3 authors had an anesthesia affiliation (Figure). When further supplemented by additional studies in these same journals known to the authors as meeting eligibility criteria, there was clear evidence of a significant increase in the successful completion and publication of such clinical trials, especially over the last 10 years (Figure). However, this systematic search also identified some additional challenges ahead.Figure.: Temporal trends in anesthesia-led publication of clinical trials in NEJM, Lancet, and JAMA from 2001 to 2021. Retracted articles were not included in this summary. JAMA indicates Journal of the American Medical Association; NEJM, New England Journal of Medicine.Several authors have opined that most randomized controlled trials (RCTs) remain underpowered and insufficiently robust.7,8 To increase the likelihood that a trial will correctly detect a treatment effect where it truly exists, many granting agencies (including the National Institutes of Health and Pharma sponsors) now require studies to be powered to 90% instead of the historically more common 80%. However, our publication analysis (Figure) shows that even in recent RCTs, “90 has not yet become the new 80” in terms of power. Furthermore, others have recommended that p-value thresholds for determining statistical significance should be lowered below the conventional cut-point criterion of 0.05,9 which will increase sample sizes for RCTs even more. By definition, larger trials are more expensive and operationally challenging. Hence, Professor Myles’ recommended approaches to increase the operational and statistical efficiency of randomized trials are critically important.3 Successful completion of large randomized trials requires the participation and cooperation of many individuals across several sites. Efforts to appropriately recognize such contributions must be enhanced, given the sometime-challenging academic environments and restrictive interpretations of journal editorial policies. If we are to succeed, building a culture of collaboration for the common good is essential, and attitudes that “I won’t (or even worse, my surgeons won’t let me) participate unless we are guaranteed masthead authorship” must be overcome. In addition, addressing the extremes of the affordability gap, whereby sites in some low-income countries cannot afford simple infrastructure for clinical research, whereas the cost of research in some high-income countries is excessive, will enhance the ability to recruit participants and enhance global generalizability of our research. However, increased sample sizes and novel trial designs do not by themselves ensure that trials are more impactful. We would argue that similar attention should be paid to the fundamental questions being asked by trials in anesthesiology and perioperative medicine. Otherwise, we risk conducting large, well-designed, expensive, and efficient trials that deliver the correct answer to the wrong question, or the wrong answer to the correct question. Notably, while large trials, with sample sizes in the thousands of patients, are increasingly common in anesthesiology and perioperative medicine, many of these large trials do not find evidence of significant benefit from the interventions being assessed. Indeed, our analysis has shown that although the number published RCTs has increased, so too has the proportion of studies with negative outcomes and results (Figure). The number of trials with negative outcomes in acute respiratory distress syndrome (ARDS) raised concerns that the clinical community would develop nihilism about efforts to improve outcomes in patients with ARDS.10,11 Evidence that an initially promising intervention is not as good as originally thought is certainly important, but such “negative” trials by themselves are unlikely to substantially improve clinical outcomes and sustain interest for participation in future RCTs. Thus, as we strive to conduct larger and more sophisticated RCTs in a more efficient manner, how might those same trials be more likely to identify interventions that improve outcomes? When selecting interventions and inclusion criteria, trial design should more closely consider the underlying pathophysiology of disease states as well as the clinical heterogeneity of patient populations. For example, patients might develop persistent postsurgical pain or acute kidney injury through a range of different mechanisms that themselves vary based on patient characteristics (eg, psychological profile, medical comorbidities, and operative characteristics). It is, therefore, unlikely that interventions targeting only a single presumed mechanism or biological pathway would result in significant overall benefits in trial samples. This problem is further accentuated when perioperative interventions are applied prophylactically to prevent postoperative adverse events like acute myocardial injury, acute kidney injury, or surgical site infection. In such trials, a large proportion of patients—often >80% (since outcome events in large trials are generally lower than 20%)—are unlikely to experience the adverse outcome of interest regardless of whether they receive the intervention or control therapy. Such individuals will never experience therapeutic benefit from the intervention and are only exposed to any of its associated adverse effects. In combination, these issues lead to trials assessing interventions with, at most, very small average treatment effects in the trial sample, thereby leading a high probability that no overall benefit would be identified. To address this problem, we recommend that researchers in anesthesiology and perioperative medicine better integrate mechanistic research to identify the most likely pathophysiological mechanisms for important disease states in that setting (eg, myocardial injury and delirium), consider trials that evaluate bundles of interventions targeting several potential causal mechanisms in combination, and adopt strategies such as those being adopted in critical care medicine to better define target populations for trials.12,13 By defining more homogenous target populations with underlying pathophysiology, which are more likely to be amenable to the intervention being tested, the possible treatment effect of the intervention is larger. A focus on better defined target populations is somewhat at tension with a focus on large pragmatic generalizable trials with very broad inclusion criteria. Generalizability is important for implementation across multiple centers and countries. However, it is likely that some perioperative interventions should be targeted to individuals with some evidence of underlying pathophysiology amenable to the intervention. Indeed, the clinical homogeneity of participants in trials targeting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pneumonia might explain, in part, why some efficacious interventions were identified (eg, dexamethasone), in contrast to the generally negative results seen in most prior trials in clinically heterogenous participants with ARDS. Some of the design approaches being assessed, like predictive enrichment14 and latent class analysis,15,16 may also help perioperative medicine researchers to identify clinically sensible subsets of patients where interventions have plausible likelihoods of beneficial effects. In addition, the use of emerging sophisticated analytic approaches like artificial intelligence, machine learning, and deep learning, and blockchain-based technologies have the potential to improve many phases of the clinical trials process—including study design, participant recruitment, randomization, intervention adherence, and data acquisition and analysis.17 The potential for these novel approaches to accelerate scientific discovery has been met with much excitement; however, they have ethical, moral, and technical implications that must be considered for addressing and improving inequity in health research and clinical care.18 Great advancements have been made in clinical trial methodology, which have enhanced the design, conduct, and reporting of perioperative research. Although we have yet to design the “perfect” clinical trial—which itself may be an impossible feat—improvements in clinical trial design provide us the opportunity to increase the efficiency, generalizability, and uptake of anesthesia research. This is an exciting time, and maintaining a positive, receptive, and adaptable attitude will be vital for continuing our ability to improve the value of research in our practice and to our patients. Through collaboration, we can continue to develop and support high-quality evidence-based recommendations for optimizing perioperative care. E DISCLOSURES Name: Duminda N. Wijeysundera, MD, PhD. Contribution: This author helped write and revise the manuscript and approve the final manuscript. Name: Nikhil Mistry, MSc. Contribution: This author helped write and revise the manuscript and approve the final manuscript. Name: C. David Mazer, MD. Contribution: This author helped write and revise the manuscript and approve the final manuscript. This manuscript was handled by: Thomas R. Vetter, MD, MPH.