Real-world experiences with generating real-world evidence: Case Studies from PCORI's pragmatic clinical Studies program

Abstract

BackgroundOver the last decade, randomized studies evaluating outcomes of health care interventions conducted in real-world settings—often termed “pragmatic trials”—have come to be seen as an important means of obtaining relevant, actionable evidence to guide health care decisions. Despite extensive writing on methodological considerations in pragmatic trial design, limited information exists regarding the practical and logistical challenges encountered in carrying out rigorous randomized evaluations in highly representative, real-world contexts. MethodsThe Patient Centered Outcomes Research Institute (PCORI) convened an expert panel in 2017 to examine common tradeoffs in study design and implementation through 3 “case studies” of in-progress, PCORI-funded pragmatic trials. This paper summarizes the findings of this panel, using the 3 examples to illustrate common implementation challenges encountered in pragmatic trials. ResultsInvestigators aimed to generate highly generalizable findings that could address real-world clinical decisions; however, practical considerations required that each study incorporate traditionally “explanatory” elements to achieve a “fit-for-purpose” approach to design and implementation. Within individual studies, efforts to balance pragmatic versus explanatory perspectives often involved multiple, diverse aspects of trial design and implementation, and the aspects of design and implementation where investigators reported encountering such tradeoffs varied across the three cases we examined. ConclusionsEfforts to generate rigorous evidence that is generalizable to “real-world” practice require continuous and iterative efforts to balance “pragmatic” and “explanatory” perspectives. In each study examined, these tradeoffs were guided both by an overriding effort to maintain pragmatism and practical considerations that varied depending on the research question and study context.