It's Time to Rethink 'Real-World Evidence': A Call for Terminological Clarity in Health Technology Assessment.

Learning from the past to advance tomorrow's real-world evidence: what demonstration projects have to teach us.

What's the Reality of Real-World Evidence in Health Technology Appraisal?

Health technology assessments and real-world evidence: tell us what you want, what you really, really want.

Real-world evidence (RWE) is increasingly considered in regulatory and health technology assessment (HTA) decision-making, though perspectives on its relevance may vary. Expanding on a recent review regarding regulatory decisions, this study aimed to identify factors influencing the need for RWE in HTA decision-making, confirm and enrich factors with stakeholder views, and evaluate similarities and differences between regulatory and HTA needs. Previous scoping review methodology was used to identify factors influencing the need for RWE in HTA decision-making. Semi-structured interviews with stakeholders were conducted to confirm and enrich literature-derived factors for both regulatory and HTA contexts. Insights from the reviews and interviews were combined to explore similarities and differences in RWE needs across these domains. The HTA review, featuring 118 articles, revealed two major themes and six subthemes, encompassing 45 factors. The need for RWE depended on (1) questions addressable with RWE, and (2) contextual factors. Stakeholder interviews confirmed literature-derived factors. While contextual factors aligned between regulatory and HTA decision-making, question-related factors partly differed. Unlike the benefit-risk assessment in regulatory decision-making, RWE serves as direct input for the HTA, and involves specific details and a broader scope. Regulators require RWE for orphan status submissions, alternative approval pathways and to evaluate the impact of risk minimization measures, whereas HTA uses RWE to guide comparator selection, evaluate treatment implementation, quality of care and general healthcare impacts. Contextual factors that influence the need for RWE are similar between regulatory and HTA decision-making, with variations seen in questions addressable with RWE.

IntroductionThe Health Technology Assessment Network of the Americas (RedETSA) working groups gather members aiming to develop projects and research jointly. The Real-World Evidence (RWE) Working Group seeks to identify methodologies and approaches allowing RWE integration into health technology assessment (HTA) in the region. This collaborative initiative is crucial to enhancing evidence-based decision-making processes in the Americas, shaping regional health policies.MethodsA survey was conducted among RedETSA members to identify RWE use in the context of HTA within the network. The survey included responses from 22 members, reflecting diverse perspectives from across the Americas. A search of official documents issued by various regulatory and HTA agencies was carried out to perform a detailed analysis of the RWE frameworks accessible online. Relevant documents published between 2013 and 2024 were selected, reviewing both the official portals of each agency and publicly accessible academic databases. Documents in English, Spanish, Portuguese, and French were searched.ResultsRWE documents from various HTA or regulatory agencies from different countries or regions were identified. The following domains were analyzed from each document: institution type, RWE definition, objective or purpose of RWE use, data quality and tools to assess RWE, data sources used, proposals for analytical methodologies, approved regulatory uses, and specific initiatives or tools. The RWE Working Group document was developed in Spanish. The document provides practical guidelines and checklists for implementing RWE, making it accessible for both specialists and general stakeholders within RedETSA.ConclusionsThe document was a helpful tool aimed at RedETSA members integrating RWE into HTA. Additional tools will include a procedure to meet regulatory and HTA expectations; and a decision tool to support a particular study design adoption. By leveraging these tools, decision-makers in the region can justify their choice of study design and align with regulatory requirements more effectively.

Objective: This review aimed to assess the current use and acceptance of real-world data (RWD) and real-world evidence (RWE) in health technology assessment (HTA) process. It additionally aimed to discern stakeholders’ viewpoints concerning RWD and RWE in HTA and illuminate the obstacles, difficulties, prospects, and consequences associated with the incorporation of RWD and RWE into the realm of HTA.Methods: A comprehensive PRISMA-based systematic review was performed in July 2022 in PubMed/Medline, Scopus, IDEAS-RePEc, International HTA database, and Centre for Reviews and Dissemination with ad hoc supplementary search in Google Scholar and international organization websites. The review included pre-determined inclusion criteria while the selection of eligible studies, the data extraction process and quality assessment were carried out using standardized and transparent methods.Results: Twenty-nine (n = 29) studies were included in the review out of 2,115 studies identified by the search strategy. In various global contexts, disparities in RWD utilization were evident, with randomized controlled trials (RCTs) serving as the primary evidence source. RWD and RWE played pivotal roles, surpassing relative effectiveness assessments (REAs) and significantly influencing decision-making and cost-effectiveness analyses. Identified challenges impeding RWD integration into HTA encompassed limited local data access, complexities in non-randomized trial design, data quality, privacy, and fragmentation. Addressing these is imperative for optimal RWD utilization. Incorporating RWD/RWE in HTA yields multifaceted advantages, enhancing understanding of treatment efficacy, resource utilization, and cost analysis, particularly via patient registries. RWE complements assessments of advanced therapy medicinal products (ATMPs) and rare diseases. Local data utilization strengthens HTA, bridging gaps when RCT data is lacking. RWD aids medical device decision-making, cancer drug reassessment, and indirect treatment comparisons. Challenges include data availability, stakeholder acceptance, expertise, and privacy. However, standardization, training, collaboration, and guidance can surmount these barriers, fostering enhanced RWD utilization in HTA.Conclusion: This study highlights the intricate global landscape of RWD and RWE acceptance in HTA. Recognizing regional nuances, addressing methodological challenges, and promoting collaboration are pivotal, among others, for leveraging RWD and RWE effectively in healthcare decision-making.

Introduction: Real-world evidence (RWE) in health technology assessment (HTA) holds significant potential for informing healthcare decision-making. A multistakeholder workshop was organised by the European Health Data and Evidence Network (EHDEN) and the GetReal Institute to explore the status, challenges, and opportunities in incorporating RWE into HTA, with a focus on learning from regulatory initiatives such as the European Medicines Agency (EMA) Data Analysis and Real World Interrogation Network (DARWIN EU®). Methods: The workshop gathered key stakeholders from regulatory agencies, HTA organizations, academia, and industry for three panel discussions on RWE and HTA integration. Insights and recommendations were collected through panel discussions and audience polls. The workshop outcomes were reviewed by authors to identify key themes, challenges, and recommendations. Results: The workshop discussions revealed several important findings relating to the use of RWE in HTA. Compared with regulatory processes, its adoption in HTA to date has been slow. Barriers include limited trust in RWE, data quality concerns, and uncertainty about best practices. Facilitators include multidisciplinary training, educational initiatives, and stakeholder collaboration, which could be facilitated by initiatives like EHDEN and the GetReal Institute. Demonstrating the impact of "driver projects" could promote RWE adoption in HTA. Conclusion: To enhance the integration of RWE in HTA, it is crucial to address known barriers through comprehensive training, stakeholder collaboration, and impactful exemplar research projects. By upskilling users and beneficiaries of RWE and those that generate it, promoting collaboration, and conducting "driver projects," can strengthen the HTA evidence base for more informed healthcare decisions.

Real-world evidence (RWE) is increasingly used and accepted by health technology assessment (HTA) bodies as supportive evidence to inform the approval of new technologies. However, the criteria driving RWE acceptance are often unclear.This study aims to improve understanding of the role and value of RWE in HTA decision-making and outline the best practices in building real-world external control arms (ECAs).A mixed approach of a targeted literature review and HTA expert interviews was applied. The HTA reports of ten selected technologies and the expert interviews from France, Germany, Italy, Spain, and the UK informed the criteria driving the acceptance of RWE. Overall, the UK and Spanish HTA bodies are more receptive to accepting RWE, whereas the French and German are the least accepting. When RWE is used to substantiate efficacy claims, the level of scrutiny from regulators and HTA bodies is considerably higher than when RWE has different intended uses. Representativeness of the data source, overall transparency in the study and robust methodologies are the key criteria driving RWE acceptance across markets.

PCN136 Evaluation of the Extent of Real-World Evidence (RWE) Used Within Health Technology Appraisals (HTA) in Oncology: A Comparative Study of Six HTA Agencies

IntroductionThe Early Access to Medicines Scheme (EAMS) aims to provide access to medicines prior to market authorization for patients with severe, life-threatening diseases who do not have adequate treatment options. An EAMS designation enables the potential collection of United Kingdom-specific real world evidence (RWE) prior to health technology assessment (HTA) by the National Institute for Health and Care Excellence (NICE). This research evaluates whether RWE is being gathered through the EAMS and utilized to support HTA submissions.MethodsAll EAMS designations as of 7 November 2018 were identified from the Medicines and Healthcare products Regulatory Agency website. For products with final NICE guidance, all publicly-available NICE documentation was reviewed.ResultsSixteen product and indication pairings with an EAMS designation were identified, with 12 having received final NICE guidance (11 were recommended, 3 were recommended for temporary reimbursement via the Cancer Drugs Fund, and 2 were not recommended). Of the 11 recommended products, seven had references to the number of patients or sites with product access through the EAMS, but only one (dupilumab for atopic dermatitis) had detailed data collected during the EAMS period. The manufacturer of dupilumab reported baseline demographics and disease characteristics from a cohort of 35 patients treated under the EAMS to inform the generalizability of trial populations for clinical practice. Follow-up results from this cohort demonstrated that real-world data on dupilumab effectiveness was comparable with the clinical trial data, despite a higher proportion of patients in the real-world cohort receiving immunosuppressant therapy, which makes improvements in efficacy harder to achieve. The committee also noted that the RWE presented supported the understanding of dupilumab's long-term clinical effectiveness and informed assumptions for the economic model.ConclusionsTo date, the majority of products receiving an EAMS designation have not presented RWE at NICE reappraisal. The case of dupilumab illustrated how RWE collected through the EAMS can be used to reduce uncertainty around how clinical trial data can be translated into clinical practice. In the future, RWE may increasingly be used to help inform NICE decisions.

Use of Real-World Evidence in Health Technology Reassessments Across 6 Health Technology Assessment Agencies.

PCN36 - New Drugs In Advanced Melanoma: Disparities In Requirements For Post-Launch Real-Word Evidence In Europe

PHP171 - Inclusion of real-world evidence in submission packages to health technology assessment bodies: What do current guidelines indicate?

IntroductionRandomized controlled trials (RCTs) are typically considered the gold standard source of clinical evidence for reimbursement submissions, but they can often be resource-intensive, expensive, and may not always be appropriate. For example, it may be unethical to assign patients to an untreated or undiagnosed control group, or blinding may not be feasible when assessing medical devices. Evidence for medical devices is therefore often limited to nonrandomized studies. We explored the use and value of real-world evidence (RWE) in the reimbursement of medical devices across several health technology assessment (HTA) agencies.MethodsA narrative review was completed to compare the acceptability of RWE for the HTA evaluation of medical devices across a convenience sample of countries. English-language published guidance documents were reviewed, and study design preferences extracted.ResultsIn Australia, France, Germany, Ireland, Norway, and Scotland, HTA agencies prefer RCT evidence but accept RWE as supporting data. In England, there is no preferred study design, with directly observed clinical outcomes, evidence syntheses, nonclinical, and modelling studies accepted. Notably, methods and processes for HTA programs are being reviewed and are expected to place a greater emphasis on RWE. In Australia, pseudo-randomized trials, comparative cohort studies, case series, and other study designs are permitted. In France, nonrandomized or nonblinded trials, patient preference cohorts, prospective comparative observational studies, and propensity score matched cohorts are permitted, accompanied by justification. In Scotland, lived experiences, RWE, and systematic reviews are accepted. In Germany, nonrandomized studies are deemed to provide “minimum”, “very low” or “low” certainty of results. In Norway, RWE may be accepted if no RCT data are available, or to support RCTs.ConclusionsIn the assessment of medical devices, where RCTs are unsuitable, RWE can form a feasible alternative. Real-world evidence is increasingly being recognized as a valuable source of evidence for medical interventions and is accepted by a number of HTA agencies. No funding was received for this study.

Real-world evidence (RWE) is being increasingly used by a wide range of stakeholders involved in the therapeutic product lifecycle but remains underutilized in the health technology assessment (HTA) process. RWE aims to fill the current evidence gaps, reduce the uncertainty around the benefits of medical technologies, and better understand the long-term impact of health technologies in real-world conditions. Despite the minimal use of RWE in some elements of HTA, there has been a larger push to further utilize RWE in the HTA processes. HTA bodies, as other stakeholders, work towards developing more robust means to leverage RWE from various data sources in the HTA processes. However, these agencies need to overcome important challenges before the broader incorporation of RWE into their routine practice. This paper aims to explore the extensive integration of RWE utilizing diverse sources of RWD. We discuss the utilization of RWE in HTA processes, considering aspects such as when, where, and how RWE can be effectively applied. Additionally, we seek the potential challenges and barriers associated with the utilization of different data sources.