From data to evidence: the evolving role of observational studies and target trial emulation in regulatory decision-making

To replicateor not to replicate? Insights and interpretations from a randomized trial duplication initiative.

Learning from the past to advance tomorrow's real-world evidence: what demonstration projects have to teach us.

Enhancing real-world data (RWD) quality in oncology: Implementation of the qcard initiative in a large US oncology electronic health record (EHR)-derived database for research and regulatory decision-making

The adoption and use of real-world evidence (RWE) is becoming increasingly important to drug development and patient safety. The Tufts Center for the Study of Drug Development (CSDD) conducted a benchmark survey of pharmaceutical and biotechnology companies and contract research organizations in a number of areas that support real-world data (RWD) and evidence, including operations and performance areas. Data were gathered on organizational functions, staff, roles and responsibilities, and skill sets required. Also, current and future allocation of budgets and spending were examined as well as return on investment measures. A total of 30 unique companies responded to the survey. Nearly all respondents (29/30 companies) reported that their organizations had an RWE function and most companies indicated that their RWE functions were increasing in size (21 companies). From a postapproval regulatory and labeling perspective, there were two primary areas for company use of RWD to generate evidence: one for postapproval safety studies, including decreasing the severity of a label warning or to support risk evaluation and mitigation strategies (REMS) (12/22 companies; 55%), which allows for real-world patient population data to inform safety decisions; and the other for postmarketing studies (13/23 companies; 57%). Developing greater insight into therapeutic area needs, gaining market access, and greater understanding of drug effectiveness were the top measures identified for return on investment for use of RWE. Expanding the use of RWE in regulatory decision making and increasing uses of real-world data by sponsors will fill the gaps that are critically needed for drug development and safety.

Background: Aligning with 21st Century Cures legislation, FDA is exploring various methodologies to advance appropriate uses of Real-World Data (RWD) to generate Real-World Evidence (RWE). Inclusion of RWD to support regulatory decision making has increased in oncology, and this review specifically focused on characterizing RWD submissions for the treatment of breast cancer (BC). Methods: A systematic search was conducted using internal FDA databases to identify RWD submissions from 2010 to 2020. Search terms included real world evidence, real world data, cancer registry, administrative claims, external control arm, and other terms relevant to RWD/RWE. Relevant regulatory submissions were reviewed, pre-defined common data elements were extracted, and the subset applicable to breast cancer was evaluated. Results: Of 142 regulatory submissions that included RWD, 6 specifically evaluated BC indications and 3 were for solid tumor indications with potential applicability to BC, corresponding to 4 new molecular entities. Regulatory objectives included support for labeling changes including efficacy (expanded indications), safety , and dose or administration modifications. The most commonly used design was a retrospective observational study with structured electronic health records (EHRs) or medical claims data, supplemented by unstructured data from medical records or chart review for missing data elements. Four of the 6 BC submissions were significantly limited by a high degree of data missingness and confounding, with some studies including key covariates that were missing in >50% of the structured data. RWD was used to provide contextual evidence for label expansion for populations not included or adequately represented in the registration trial. Of note, for the application expanding the label to include treatment of male BC, the regulatory decision was primarily based on clinical trial data. The primary rwEndpoints submitted were overall survival (rwOS), progression free survival (rwPFS), response rate (rwORR) and time to next treatment (TTNT). Safety outcomes were investigated in all but 1 of the studies, most commonly as a secondary RWD endpoint. Conclusion: In our review of regulatory submissions relevant to breast cancer therapies, RWD has largely been used to contextualize and complement prospective clinical trial data. Evaluating that selected RWD is fit for purpose to address the regulatory objective(s) and all analytical plans are prespecified allows for robust data characterization, and appropriate evaluation. Data relevance (availability of key variables) along with reliability assessment which includes evaluating data for completeness, consistency, and trends over time are necessary for the rigorous evaluation of RWE in drug development. Data missingness is a key issue in RWD, especially when structured data are not available and specific variables are unlikely to be captured in a reliable way in the unstructured data or further validation is not feasible. To optimize RWD as evidence for specific patient populations, attention to the proportion of patients excluded is necessary to avoid concerns regarding the generalizability of the data. Careful selection of rwEndpoints must be aligned with the study design and objective, include data such as prior, concomitant and subsequent anti-cancer treatments, and the ability for outcome validation to be methodologically appropriate. When contemplating a regulatory submission using RWD, early consultation with the appropriate FDA review division can provide additional feedback on the appropriate use of RWD or pragmatic designs. Citation Format: Melanie E Royce, Jennifer J. Lee, Christy L. Osgood, Laleh Amiri-Kordestani, Julia A. Beaver, Paul G. Kluetz, Donna R. Rivera. Methodological approaches to the use of real-world data(RWD) for medical products to treat breast cancer: An FDA oncology center of excellence evaluation of RWD submissions [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P5-19-02.

Lately, the use of Real-World Data (RWD)/Real-World Evidence (RWE) have been actively expanded in regulatory decisions. We aimed to investigate how RWD/RWE were used in pursuit of supporting regulatory actions of FDA and EMA. We selected 9 examples of regulatory decision making with the support of RWD/RWE. To identify information included in RWE submission, we referred to the official website of US FDA or EMA. In addition, we used ClinicalTrials.gov and Medline database to collect information on study design using RWE and RWD source used to generate RWE. Our study drugs were classified into 3 parts according to regulatory context; primary approval, label expansion, and regulatory response to safety signal after marketing. Among these examples, 5 biologic products including 1 vaccine had gone through accelerated review supported by RWE. To summarize, we investigated several examples where RWD/RWE has already been used in the process of making regulatory decisions for assessing the safety of drugs as well as their effectiveness. We recommend further discussions on advancing the use of RWD/RWE in regulatory context to improve patient access to novel rare disease treatments and provide more efficient safety assessment.

Real-World Data in the Postapproval Setting as Applied by the EMA and the US FDA

The 21st Century Cures Act of 2016 provided a framework to the US Food and Drug Administration (FDA) to rapidly move treatments to patients.1 The increased acceptability of real‐world data (RWD) sources allows for innovative ways to study products and has the potential to reduce trial costs. Published papers provide guidance regarding data quality issues, reproducibility, and validity assessment.2 Rapid evolvement of electronic health records (EHRs) encourages greater consideration of their use in research.1, 2, 3, 4, 5, 6 For years, the FDA has relied on epidemiological studies of postapproval product safety using RWD5, 6 (eg, administrative claims and EHR) and for device effectiveness studies4; however, regulatory use for evaluating drug effectiveness has been rare. As part of the Prescription Drug User Fee Act (PDUFA VI),3 use of RWD is being considered for potential contributions to evaluating effectiveness and safety of new indications for approved products and to satisfy postapproval study requirements. Recently, the Duke Margolis Center for Health Policy held workshops and issued two paper on this topic.5, 6 The first paper focused on defining RWD as data routinely collected pertinent to patient health status and/or delivery of care, and the use of RWD in regulatory and clinical contexts.5 The second white paper from the October 1, 2018, workshop focused on data relevancy and quality, including cleaning, transforming, and linking RWD to characterize RWD sources as “fit for regulatory purpose.”6 These papers offer a practical “commonsense” high‐level view of primary data and methods considerations for RWD use from a regulatory perspective, facilitating discussion around regulatory uses of RWD within the research community and industry. However, salient points are missing from the papers and the RWD discussions among FDA, researchers, and industry. Here, we provide a commentary on the data considerations discussed in the white papers and highlight pertinent considerations with respect to RWD in the context of whether data are relevant, representative, and robust. 1.1. Data relevance The recent white paper defines data relevance dimensions including representativeness of the population of interest, critical data field availability, accurate linking at the patient level with multiple data sources, and adequate sample size and follow‐up time to demonstrate expected treatment effects.6 Guidance from FDA on how to ensure RWD are fit for purpose and adequate to support regulatory decisions would be helpful on each dimension. Determining if RWD is fit for regulatory purpose is a “contextual exercise” where the specific research question, regulatory use, and data characteristics drive what meaningful conclusions can be drawn.6 Covariates may be critical for one research question but not another. Exposures and outcomes should be well defined when part of the research question but may not be critical for natural history studies. There is no “one‐size‐fits‐all” approach, and critical data components should be evaluated for each research question and regulatory use.7 A framework is needed to guide choice and evaluation of critical data elements for specific research questions for regulatory use. Representativeness of the population of interest is gauged in many ways. Recent FDA guidance on Patient Focused Drug Development suggests a statistical sampling approach be used to obtain patient experience data representative of the target population.8 However, most US real‐world databases use administrative claims or EHR for patients seeking medical attention. These RWD sources should be considered broadly representative of the population eligible for using most, if not all, new products and services. “Representativeness” should be assessed broadly in the context of likely product users with some diversity in geography, health status, and health care system as appropriate for the specific research question and regulatory context. While data linkage is likely to limit the eligible sample, it may be needed to increase the informative nature of RWD, especially with increasing evaluations to support precision medicine. Sample size should be derived based on anticipated treatment effects for studies of treatment effectiveness or safety, whether comparative or not, to ensure appropriate precision of estimates. For rare diseases, there should be flexibility given data sparseness worldwide, as indicated in the FDA guidance on rare disease.8 Additional guidance would be useful regarding how “accurate linking” should be assessed since linking 100% of patients with administrative claims and EHR is impractical. Would FDA accept limited linked data if it was supplemental to cruder variables in the full dataset? Would a subset of 60% be adequate? In the context of probabilistic linkage, what level of certainty would constitute adequate linkage? Salience of linkable individuals to the specific research question should be considered in this determination and pre‐specified sensitivity analyses should help assess robustness of results and conclusions.9, 10

Following the release of the framework for the Real-World Evidence (RWE) Program, the US Food and Drug Administration (FDA) is actively evaluating and exploring ways to optimize the utility of real-world data (RWD) and RWE to support regulatory decision making. For rare conditions, conducting traditional randomized clinical trials may not always be feasible, and RWD and RWE have played and will continue to play an important role. We use three case examples-cerliponase alfa, asfotase alfa, and uridine triacetate-to illustrate how RWD from disease registries, medical records with chart review, and literature, respectively, have been used to generate RWE to support regulatory decisions for selected rare diseases. These examples highlight the need for improving data reliability and quality in existing data to expand use of RWD and RWE beyond "hard endpoints" and standardizing data collection for outcome measures in patient registries to expand its utility. We also discuss a recent FDA guidance for using RWE in supporting rare disease drug development, including its recommendations about using natural history studies as external control groups for single-arm interventional trials. The external control group needs to be comparable with the treated group. Selection bias and confounding are major concerns because of lack of randomization and unrecognized baseline differences. Use of valid epidemiological approaches can reduce these biases. Lastly, we discuss future directions to expand the use of RWD and RWE to support orphan drug approvals, including the need for including patient experience data as an important source of RWD.

Putting Theory to the Test.

6611 Background: The 21st Century Cures Act in 2016 enhanced the FDA’s ability to include the use of real-world data (RWD) and consideration of RWE in the development and approval of new medical products. Currently, limited assessments of the use of RWE in FDA approved oncology product reviews exist in the literature. This study was conducted to determine how RWE was used in FDA decisions for oncology drug approvals between 2015 and 2022. Methods: A systematic review of FDA submission documents and product labels available on the FDA website from 2020 to 2022, in addition to an exhaustive manual search of grey literature and the results of a targeted literature search from 2015 to 2019 were used to identify RWE that were included in FDA submission documents and in approved product labels. Results: In total, 124 oncology product evaluations were reviewed in the 2020 to 2022 literature search. Twenty-nine oncology product NDAs and BLAs included RWE to support efficacy, safety, or indication, of which the majority (n=26) ultimately had RWE accepted as evidence by the FDA for the submission. In these cases, RWE was used to add context (n=16), to support the indication or safety (n=7), to provide substantial basis for indication or safety (n=2), as a statistical comparison (n=2), or for informational purposes only (n=1). Feedback from the FDA reviewers for reasons RWE studies were not considered during product approval reviews included methodology issues, sample size concerns, and omission of patient level data. Ultimately, across the 2015 to 2019 targeted literature review and the 2020 to 2022 FDA submission document review, 6 oncology product labels incorporated RWE: palbociclib, uridine triacetate, lutetium Lu 177 DOTATATE, bevacizumab-adcd, naxitamab-gqgk, and decitabine/cedazuridine. RWE was used to support the initial indication for the majority of products (n=5) and also supported label expansion (n=1). RWE was incorporated in the product label postmarketing experience section (n=3), the clinical studies section (n=2), and the warnings/precautions section (n=1). Conclusions: These results suggest that RWE has become an important part of regulatory decision-making for oncology drug approval. Twenty-six of 29 RWE studies were accepted as evidence for the submission. For the 6 oncology products that include RWE in the product label, RWE was most frequently used to support the initial indication and was included in the postmarketing experience section. These examples show consistent use of RWE since the approval of the 21st Century Cares Act and provide evidence of a promising new direction with the creation of the FDA guideline/program for RWE.

Commentary on Point-of-Care Clinical Trials in Sports Medicine Research: Identifying Effective Treatment Interventions Through Real-World Evidence.

Real-world data (RWD) and real-world evidence (RWE) have been increasingly used in medical product development and regulatory decision-making, especially for rare diseases. After outlining the challenges and possible strategies to address the challenges in rare disease drug development (see the accompanying paper), the Real-World Evidence (RWE) Scientific Working Group of the American Statistical Association Biopharmaceutical Section reviews the roles of RWD and RWE in clinical trials for drugs treating rare diseases. This paper summarizes relevant guidance documents and frameworks by selected regulatory agencies and the current practice on the use of RWD and RWE in natural history studies and the design, conduct, and analysis of rare disease clinical trials. A targeted learning roadmap for rare disease trials is described, followed by case studies on the use of RWD and RWE to support a natural history study and marketing applications in various settings.

In recent years, with the development of big data application technology, the real-world data and the corresponding generated real-world evidence have attracted the attention of healthcare regulatory authorities around the world. Regulators recognize that real-world research with specific purposes using real-world data can provide important evidence for regulatory decisions. A total of 90 instances of publicly released on the application of real-world evidence to support regulatory decisions of U. S. Food and Drug Administration are explored, and the positioning and value of real-world evidence in U. S. Food and Drug Administration regulatory decisions are summarized and analyzed, providing references for the use of real-world data and real-world evidence to promote medical devices whole cycle regulation in China.

Use of Real-world Data for New Drug Applications and Line Extensions