Real-world Evidence Generation Research Articles

Use of transportability methods for real-world evidence generation: a review of current applications.

Aim: To evaluate how transportability methods are currently used for real-world evidence (RWE) generation to inform good practices and support adoption and acceptance of these methods in the RWE context. Methods: We conducted a targeted literature review to identify studies that transported an effect estimate of the clinical effectiveness or safety of a biomedical exposure to a target real-world population. Records were identified from PubMed-indexed articles published any time before 25 July 2023 (inclusive). Two reviewers screened abstracts/titles and reviewed the full text of candidate studies to identify the final set of articles. Data on the therapeutic area, exposure(s), outcome(s), original and target populations and details of the transportability analysis (e.g., analytic method used, estimate transported, stated assumptions) were abstracted from each article. Results: Of 458 unique records identified, six were retained in the final review. Articles were published during 2021-2023, focused on the US/Canada context, and covered a range of therapeutic areas. Four studies transported an RCT effect estimate, while two transported effect estimates derived from real-world data. Almost all articles used weighting methods to transport estimates. Two studies discussed all transportability assumptions, and one evaluated the likelihood of meeting all assumptions and the impact of potential violations. Conclusion: The use of transportability methods for RWE generation is an emerging and promising area of research to address evidence gaps in settings with limited data and infrastructure. More transparent and rigorous reporting of methods, assumptions and limitations may increase the use and acceptability of transportability for producing robust evidence on treatment effectiveness and safety.

FOUNTAIN: a modular research platform for integrated real-world evidence generation

BackgroundReal-world evidence (RWE) plays a key role in regulatory and healthcare decision-making, but the potentially fragmentated nature of generated evidence may limit its utility for clinical decision-making. Heterogeneity and a lack of reproducibility in RWE resulting from inconsistent application of methodologies across data sources should be minimized through harmonization.MethodsThis paper’s aim is to describe and reflect upon a multidisciplinary research platform (FOUNTAIN; FinerenOne mUlti-database NeTwork for evidence generAtIoN) with coordinated studies using diverse RWE generation approaches and explore the platform’s strengths and limitations. With guidance from an executive advisory committee of multidisciplinary experts and patient representatives, the goal of the FOUNTAIN platform is to harmonize RWE generation across a portfolio of research projects, including research partner collaborations and a common data model (CDM)–based program. FOUNTAIN’s overarching objectives as a research platform are to establish long-term collaborations among pharmacoepidemiology research partners and experts and to integrate diverse approaches for RWE generation, including global protocol execution by research partners in local data sources and common protocol execution in multiple data sources through federated data networks, while ensuring harmonization of medical definitions, methodology, and reproducible artifacts across all studies. Specifically, the aim of the multiple studies run within the frame of FOUNTAIN is to provide insight into the real-world utilization, effectiveness, and safety of finerenone across its life-cycle.ResultsCurrently, the FOUNTAIN platform includes 9 research partner collaborations and 8 CDM-mapped data sources from 7 countries (United States, United Kingdom, China, Japan, The Netherlands, Spain, and Denmark). These databases and research partners were selected after a feasibility fit-for-purpose evaluation. Six multicountry, multidatabase, cohort studies are ongoing to describe patient populations, current standard of care, comorbidity profiles, healthcare resource use, and treatment effectiveness and safety in different patient populations with chronic kidney disease and type 2 diabetes. Strengths and potential limitations of FOUNTAIN are described in the context of valid RWE generation.ConclusionThe establishment of the FOUNTAIN platform has allowed harmonized execution of multiple studies, promoting consistency both within individual studies that employ multiple data sources and across all studies run within the platform’s framework. FOUNTAIN presents a proposal to efficiently improve the consistency and generalizability of RWE on finerenone.

Addressing immortal time bias in precision medicine: Practical guidance and methods development.

To compare theoretical strengths and limitations of common immortal time adjustment methods, propose a new approach using multiple imputation (MI), and provide practical guidance for using MI in precision medicine evaluations centered on a real-world case study. Methods comparison, guidance, and real-world case study based on previous literature. We compared landmark analysis, time-distribution matching, time-dependent analysis, and our proposed MI application. Guidance for MI spanned (1) selecting the imputation method; (2) specifying and applying the imputation model; and (3) conducting comparative analysis and pooling estimates. Our case study used a matched cohort design to evaluate overall survival benefits of whole-genome and transcriptome analysis, a precision medicine technology, compared to usual care for advanced cancers, and applied both time-distribution matching and MI. Bootstrap simulation characterized imputation sensitivity to varying data missingness and sample sizes. Case study used population-based administrative data and single-arm precision medicine program data from British Columbia, Canada for the study period 2012 to 2015. While each method described can reduce immortal time bias, MI offers theoretical advantages. Compared to alternative approaches, MI minimizes information loss and better characterizes statistical uncertainty about the true length of the immortal time period, avoiding false precision. Additionally, MI explicitly considers the impacts of patient characteristics on immortal time distributions, with inclusion criteria and follow-up period definitions that do not inadvertently risk biasing evaluations. In the real-world case study, survival analysis results did not substantively differ across MI and time distribution matching, but standard errors based on MI were higher for all point estimates. Mean imputed immortal time was stable across simulations. Precision medicine evaluations must employ immortal time adjustment methods for unbiased, decision-grade real-world evidence generation. MI is a promising solution to the challenge of immortal time bias.

Data Science Methods for Real-World Evidence Generation in Real-World Data.

In the healthcare landscape, data science (DS) methods have emerged as indispensable tools to harness real-world data (RWD) from various data sources such as electronic health records, claim and registry data, and data gathered from digital health technologies. Real-world evidence (RWE) generated from RWD empowers researchers, clinicians, and policymakers with a more comprehensive understanding of real-world patient outcomes. Nevertheless, persistent challenges in RWD (e.g., messiness, voluminousness, heterogeneity, multimodality) and a growing awareness of the need for trustworthy and reliable RWE demand innovative, robust, and valid DS methods for analyzing RWD. In this article, I review some common current DS methods for extracting RWE and valuable insights from complex and diverse RWD. This article encompasses the entire RWE-generation pipeline, from study design with RWD to data preprocessing, exploratory analysis, methods for analyzing RWD, and trustworthiness and reliability guarantees, along with data ethics considerations and open-source tools. This review, tailored for an audience that may not be experts in DS, aspires to offer a systematic review of DS methods and assists readers in selecting suitable DS methods and enhancing the process of RWE generation for addressing their specific challenges.

Bridging Real-World Data Gaps: Connecting Dots Across 10 Asian Countries.

The economic trend and the health care landscape are rapidly evolving across Asia. Effective real-world data (RWD) for regulatory and clinical decision-making is a crucial milestone associated with this evolution. This necessitates a critical evaluation of RWD generation within distinct nations for the use of various RWD warehouses in the generation of real-world evidence (RWE). In this article, we outline the RWD generation trends for 2 contrasting nation archetypes: "Solo Scholars"-nations with relatively self-sufficient RWD research systems-and "Global Collaborators"-countries largely reliant on international infrastructures for RWD generation. The key trends and patterns in RWD generation, country-specific insights into the predominant databases used in each country to produce RWE, and insights into the broader landscape of RWD database use across these countries are discussed. Conclusively, the data point out the heterogeneous nature of RWD generation practices across 10 different Asian nations and advocate for strategic enhancements in data harmonization. The evidence highlights the imperative for improved database integration and the establishment of standardized protocols and infrastructure for leveraging electronic medical records (EMR) in streamlining RWD acquisition. The clinical data analysis and reporting system of Hong Kong is an excellent example of a successful EMR system that showcases the capacity of integrated robust EMR platforms to consolidate and produce diverse RWE. This, in turn, can potentially reduce the necessity for reliance on numerous condition-specific local and global registries or limited and largely unavailable medical insurance or claims databases in most Asian nations. Linking health technology assessment processes with open data initiatives such as the Observational Medical Outcomes Partnership Common Data Model and the Observational Health Data Sciences and Informatics could enable the leveraging of global data resources to inform local decision-making. Advancing such initiatives is crucial for reinforcing health care frameworks in resource-limited settings and advancing toward cohesive, evidence-driven health care policy and improved patient outcomes in the region.

Which heart failure patients benefit most from non-invasive telemedicine? An overview of current evidence and future directions.

Telemedicine in heart failure (HF) management may positively impact health outcomes, but varied effects in studies hinder guidance in HF guidelines. Evidence on the effectiveness of telemedicine in HF subpopulations is limited. We conducted ascoping review to evaluate and synthesise evidence on the effectiveness of telemedicine across HF subpopulations that could guide telemedicine strategies in routine practice. Meta-analyses concerning randomised controlled trials (RCTs) with subgroup analyses on telemedicine effectives were identified in PubMed. We identified 15RCTs, encompassing 21different subgroups based on characteristics of HF patients. Findings varied across studies and no definite evidence was found about which patients benefit most from telemedicine. Subgroup definitions were inconsistent, not always apriori defined and subgroups contained few patients. Some studies found heterogeneous effects of telemedicine on mortality and hospitalisation across subgroups defined by: New York Heart Association (NYHA) classification, previous HF decompensation, implantable device, concurrent depression, time since hospital discharge and duration of HF. Patients represented in the RCTs were mostly male, aged 65-75years, with HF with reduced ejection fraction and NYHA classII/III. Traditional RCTs have not been able to provide clinicians with guidance; continuous real-world evidence generation could enhance monitoring and identify who benefits from telemedicine.

Characterizing Myasthenia Gravis Symptoms, Exacerbations, and Crises From Neurologist's Clinical Notes Using Natural Language Processing.

Background Myasthenia gravis (MG) is a rare, autoantibody neuromuscular disorder characterized by fatigable weakness. Real-world evidence based on administrative and structured datasets regarding MG may miss important details related to the clinical encounter. Examination of free-text clinical progress notes has the potential to illuminate aspects of MG care. Objective The primary objective was to examine and characterize neurologist progress notes in the care of individuals with MG regarding the prevalence of documentation of clinical subtypes, antibody status, symptomatology, and MG deteriorations, including exacerbations and crises.The secondary objectives were to categorize MG deteriorations into practical, objective states as well as examine potential sources of clinical inertia in MG care. Methods We performed a retrospective, cross-sectional analysis of de-identified neurologist clinical notes from 2017 to 2022. A qualitative analysis of physician descriptions of MG deteriorations and a discussion of risks in MG care (risk for adverse effects, risk for clinical decompensation, etc.) was performed. Results Of the 3,085 individuals with MG, clinical subtypes and antibody status identified included gMG (n = 400; 13.0%), ocular MG (n = 253; 8.2%), MG unspecified (2,432; 78.8%), seropositivity for acetylcholine receptor antibody (n = 441; 14.3%), and MuSK antibody (n = 29; 0.9%). The most common gMG manifestations were dysphagia (n = 712; 23.0%), dyspnea (n = 626; 20.3%), and dysarthria (n = 514; 16.7%). In MG crisis patients, documentation of difficulties with MG standard therapies was common (n = 62; 45.2%). The qualitative analysis of MG deterioration types includes symptom fluctuation, symptom worsening with treatment intensification, MG deterioration with rescue therapy, and MG crisis. Qualitative analysis of MG-related risks included the toxicity of new therapies and concern for worsening MG because of changing therapies. Conclusions This study of neurologist progress notes demonstrates the potential for real-world evidence generation in the care of individuals with MG. MG patients suffer fluctuating symptomatology and a spectrum of clinical deteriorations. Adverse effects of MG therapies are common, highlighting the need for effective, less toxic treatments.

Real-World Evidence to Support EU Regulatory Decision Making-Results From a Pilot of Regulatory Use Cases.

Studies using real-world data (RWD) can complement evidence from clinical trials and fill evidence gaps during different stages of a medicine's lifecycle. This review presents the experience resulting from the European Medicines Agency (EMA) pilot to generate RWE to support evaluations by EU regulators and down-stream decision makers from September 2021 to February 2023. A total of 61 research topics were identified for RWE generation during this period, covering a wide range of research questions, primarily generating evidence on medicines safety (22, 36%), followed by questions on the design and feasibility of clinical trials (11, 18%), drug utilization (10, 16%), clinical management (10, 16%), and disease epidemiology. A significant number of questions were related to the pediatric population and/or rare diseases. A total of 27 regulatory-led RWD studies have been conducted. Most studies were descriptive and aimed at estimating incidence and prevalence rates of clinical outcomes including adverse events or to evaluate medicines utilization. The review highlights key learnings to guide further efforts to enable the use and establish the value of real-world evidence (RWE) for regulatory decisions. For instance, there is a need to access additional fit-for-purpose and representative data, and to explore further means to provide timely evidence that meets regulatory timelines. The need for early interactions and close collaboration with study requesters, e.g., from the Agency's scientific Committees, to better understand the research question is equally important. Finally, the review provides our perspective on the way forward to maximize the potential of regulatory-led RWE generation.

Unstructured EMR data hold the key to oncology studies: Methods for validating NLP-extracted phenotypes from EMR notes.

11133 Background: Natural language processing (NLP) parsers are advanced algorithms designed to parse values of interest from free-text stored in Electronic Medical Records (EMR). Given the importance of information stored in free-text for use in oncology research, it is crucial to understand how accurately NLP algorithms are extracting oncology-specific measures of interest. We focus on validating NLP-parsers that identify three measures critical to oncology research by evaluating how closely the NLP-parser results match manual chart abstractor review: AJCC summary cancer stage group (cancer stage), AJCC TNM stage (TNM stage), and surgical treatment of cancer (surgery). Methods: Following deployment of the parsers on 8,000 non-identifiable free-text notes housed in an aggregated non-identifiable U.S. EMR database, manual chart abstraction was performed on a random selection of notes (n=300) dated between 2008 to 2023 to validate the parsers’ accurate capture of cancer stage, TNM stage, and surgical treatment of cancer. We report true positives (TP), false positives (FP) and positive percent value (PPV). Results: The PPV of AJCC Cancer Stage was 99.0% (n=297 TP, n=3 FP). The PPV of the TNM stage NLP-parser was 97.3% for tumor stage (n=292 TP, n=8 FP), 99.0% for nodal stage (n=297 TP, n=3 FP), and 96.7% for presence of metastases (n=290 TP, n=10 FP). The PPV of the surgery parser was 87.8% (n=468 TP, n=65 FP). Conclusions: These NLP-parsers performed very well identifying the measures of interest, providing confidence in use of these derived measures which are central to oncology research. Validation is a necessary initial step in processing real-world data for use in real-world evidence generation, with unique considerations needed in the oncology research space due to key information being documented in EMR free-text rather than structured data fields.

Automating the detection of treatment progression in patients with lung cancer using large language models.

e13620 Background: Accurate assessment of treatment response is fundamental to advancing cancer patient care, particularly in lung cancer, where treatment modalities are diverse and complex. The extraction of treatment response, particularly disease progression information, from Electronic Health Records (EHRs) is essential for several reasons: It facilitates generation of real-world evidence from real-world data, enables personalized treatment planning, and contributes to a broader understanding of cancer therapeutics. However, much of treatment response information is documented in free-text EHR notes and traditional methods of extracting such information from notes are labor-intensive, error-prone, and inefficient, presenting significant barriers to timely and accurate real-world evidence studies. The advancement of Natural Language Processing (NLP) technologies, especially Large Language Models (LLMs), presents a transformative opportunity to automate the extraction of treatment responses. Methods: We used a cohort of 1,953 primary lung cancer patients identified from UPMC Hillman Cancer Center cancer registry and retrieved ~113,000 clinical notes from UPMC clinical data warehouse. We focused on extracting disease progression information, following the RECIST guidelines which define progression as an increase in tumor size or cancer markers after therapy. We selected ~50 notes to perform manual annotations by an experienced oncologist and created a gold standard dataset to validate the NLP models. We fine-tuned a state-of-the-art open-source LLM named LLAMA-2 and compared against a traditional rule-based NLP system for the automated extraction of disease progression from notes. The process of fine-tuning involved adjusting the model's parameters specifically to improve its ability to recognize and classify instances of disease progression. Results: Our analysis demonstrated a significant enhancement in performance with the LLM compared to the traditional rule-based NLP system. LLM exhibited a remarkable increase in sensitivity by ~37%, indicating its superior ability to accurately identify disease progression. Additionally, the model maintained high specificity and PPV, achieving scores nearly comparable to the rule-based system but with a notable improvement in the F1-score by ~14%. Conclusions: Our research highlights the transformative potential of LLM-based NLP algorithms in automating the extraction of treatment responses from free-text EHRs. This methodology not only provides a scalable and efficient mechanism for processing large volumes of clinical text but also significantly enhances the accuracy of lung cancer treatment response assessments. [Table: see text]

Overcoming challenges in real-world evidence generation: An example from an Adult Medical Care Coordination program.

The Adult Medical Care Coordination program ("the program") was implemented at Mayo Clinic to promote patient self-management and improve 30-day unplanned readmission for patients with high risk for readmission after hospital discharge. This study aimed to evaluate the impact of the program compared to usual care using a pragmatic, stepped wedge cluster randomized trial ("stepped wedge trial"). However, several challenges arose including large differences between the study arms. Our goal is to describe the challenges and present lessons learned on how to overcome such challenges and generate evidence to support practice decisions. We describe the challenges encountered during the trial, the approach to addressing these challenges, and lessons learned for other learning health system researchers facing similar challenges. The trial experienced several challenges in implementation including several clinics dropping from the study and care disruptions due to COVID-19. Additionally, there were large differences in the patient population between the program and usual care arms. For example, the mean age was 76.8 for the program and 68.1 for usual care. Due to these differences, we adapted the methods using the propensity score matching approach that is traditionally applied to observational designs and adjusted for differences in observable characteristics. When conducting pragmatic research, researchers will encounter factors beyond their control that may introduce bias. The lessons learned include the need to weigh the tradeoffs of pragmatic design elements and the potential value of adaptive designs for pragmatic trials. Applying these lessons would promote the successful generation of evidence that informs practice decisions.

OHDSI-compliance: a set of document templates facilitating the implementation and operation of a software stack for real-world evidence generation.

The open-source software offered by the Observational Health Data Science and Informatics (OHDSI) collective, including the OMOP-CDM, serves as a major backbone for many real-world evidence networks and distributed health data analytics platforms. While container technology has significantly simplified deployments from a technical perspective, regulatory compliance can remain a major hurdle for the setup and operation of such platforms. In this paper, we present OHDSI-Compliance, a comprehensive set of document templates designed to streamline the data protection and information security-related documentation and coordination efforts required to establish OHDSI installations. To decide on a set of relevant document templates, we first analyzed the legal requirements and associated guidelines with a focus on the General Data Protection Regulation (GDPR). Moreover, we analyzed the software architecture of a typical OHDSI stack and related its components to the different general types of concepts and documentation identified. Then, we created those documents for a prototypical OHDSI installation, based on the so-called Broadsea package, following relevant guidelines from Germany. Finally, we generalized the documents by introducing placeholders and options at places where individual institution-specific content will be needed. We present four documents: (1) a record of processing activities, (2) an information security concept, (3) an authorization concept, as well as (4) an operational concept covering the technical details of maintaining the stack. The documents are publicly available under a permissive license. To the best of our knowledge, there are no other publicly available sets of documents designed to simplify the compliance process for OHDSI deployments. While our documents provide a comprehensive starting point, local specifics need to be added, and, due to the heterogeneity of legal requirements in different countries, further adoptions might be necessary.

Expert recommendation for real-world study on the application of human papillomavirus vaccine in the Yangtze River Delta region

Cervical cancer has become a global public health problem that poses a serious threat to women's health. Human papillomavirus (HPV) vaccination is an important primary preventive measure for cervical cancer. With the accelerated application of the HPV vaccine in China, conducting real-world studies (RWS) on the application of HPV vaccines can guide the rational use of the vaccine, which is of great significance for the strategy and practice of accelerating the elimination of cervical cancer in China. The Yangtze River Delta Cervical Cancer and Female Lower Genital Tract Infection Prevention and Control Alliance and Yangtze River Delta Immunization Integration Working Group organized an expert group to develop the "Expert recommendation for real-world study on the application of HPV vaccine in the Yangtze River Delta region". Based on the current situation of medical informatization and HPV vaccination administration in China, as well as literature and cases from domestic and international studies, this study describes the data chain of HPV vaccination and evaluation, introduces the study design, study direction, database construction, data governance and statistical analysis of RWS, and provides expert recommendation on the problems in carrying out the practice of HPV vaccine RWS, aiming to guide the RWS of HPV vaccine application and promote the generation of real-world evidence (RWE) in China.

Maximizing the Value of Real-World Data and Real-World Evidence to Accelerate Healthcare Transformation in China: Summary of External Advisory Committee Meetings.

Use of real-world data (RWD) is gaining wide attention. To bridge the gap between diverse healthcare stakeholders and to leverage the impact of Chinese real-world evidence (RWE) globally, a multi-stakeholder External Advisory Committee (EAC) and EAC meetings were initiated, aiming to elucidate the current and evolving RWD landscape in China, articulate the values of RWE in ensuring Chinese patients' equitable access to affordable medicines and solutions, and identify strategic opportunities and partnerships for expansion of RWE generation in China. Chinese and international experts who are clinicians and academic researchers were selected as EAC members based on their professional background and familiarity with RWD/RWE. Three EAC meetings were held quarterly in 2023. Various topics were presented and discussed for insights and suggestions. Nine experts from China, one from South Korea, and two from Europe were selected as EAC members and attended these meetings. Experts' presentations were summarized by theme, including the RWD landscape and RWE enablement in China, as well as global development of a patient-centric ecosystem. Experts' insights and suggestions on maximizing the RWD/RWE value to accelerate healthcare transformation in China were collected. We concluded that though data access, sharing, and quality are still challenging, RWD is developing to support evidence generation in the medicinal product lifecycle, inform clinical practice, and empower patient management in China. RWD/RWE creates value, accelerates healthcare transformation, and improves patient outcomes. Fostering a patient-centric ecosystem across healthcare stakeholders and maintaining global partnerships and collaboration are essential for unlocking the power of RWD/RWE.

The Use of Real-World Evidence for Regulatory Decisions in China.

There is a growing demand for the use of high-quality real-world evidence (RWE) to support regulatory decision-making worldwide and in China, which highlights the need for conducting literature reviews to evaluate the available data and evidence. This study aims to review the use of RWE in Chinese regulatory decisions and to summarize relevant regulatory and methodological considerations to inform the future use of RWE in China. We identified policy documents, technical guidance documents, and cases on official Chinese government websites and extracted their contents separately. We consulted experts from the National Medical Products Administration (NMPA) and academic institutes and searched case-related articles for enrichment. We also searched and included articles related to the use of RWE/Real-world data in Chinese regulatory decisions. Six trial versions of technical guidance documents, 7 case studies, and 40 articles related to the Chinese regulatory decisions were included in this study. Based on the technical guidance, data quality, and appropriate study design and statistical analysis are the main concerns for RWE generation. The cases and articles related to regulatory decisions revealed 9 main concerns, including data sources and applicability, data quality, strength of existing evidence, appropriate study design and statistical analysis, regulated and transparent process for analysis and evidence generation, product safety and efficacy, product characteristics and clinical needs, ethical considerations and data security, and communicate adequately with regulatory authorities. Among these concerns, data issues are central. Preliminary attempts have been made by the NMPA to promote the use of RWE, but substantial challenges still remain.

Expert survey on real-world data utilization and real-world evidence generation for regulatory decision-making in drug lifecycle in Korea.

As the importance of utilizing real-world data (RWD)/real-world evidence (RWE) for supporting regulatory scientific decision-making continues to grow, experiences and inputs from experts become crucial for developing a systematic and practice-oriented plan for the use of fit-for-purpose RWD/RWE. This study aimed to survey relevant experts from government agencies, industries, and academia to identify prerequisites for the drug life cycle in Korea. The questionnaire comprised the following: (A) the definition and categories of RWD/RWE, (B) the suitability and feasibility of using RWD/RWE at each authorization stage by the types of RWD, and (C) the challenges and solutions for the use of RWD/RWE. A total of 46 respondents completed the online survey, with 89.1% of them having prior experience with RWD/RWE usage. A majority of respondents agreed that RWD can be obtained from various sources. Among these sources, the registry was the most suitable source. It is suitable to compensate for the limitations of randomized control trials and ensure quality in data collection. Though there was consensus among the respondents for the use of RWD/RWE in post-marketing surveillance, the use of such data in new drug application (NDA) was disagreeable. Respondents considered it necessary to write a protocol in advance for RWD collection and RWE generation, for all RWD types. In conclusion, this study examined the perceptions of experts for RWD/RWE use at each approval stage of drugs. The results suggest that guidelines for the fit-for-purpose use of RWD/RWE should be developed via careful deliberation among experts in the future.

Abstract 926: Adaptive universal platform for real-world observational studies (AUPROS): An emerging model for clinical, epidemiologic, and precision oncology research

Abstract Background: Adaptive Universal Platform for Real-world Observational Studies (AUPROS) is an emerging study design and platform for the generation of real-world evidence (RWE) particularly in malignancies with rare molecular aberrations. Here, we evaluate the efficiency of AUPROS as an innovative model for real-world observational studies in oncology. Methods: We conducted a mixed-methods study to evaluate three AUPROS studies at Princess Margaret Cancer Centre either locally or as a part of national and international collaborations: 1) METAL (Molecular Epidemiology of ThorAcic Lesions; n=11,378), 2) THANKS (Translational Head And NecK Study; n=3,506), and 3) CARMA (Canadian Cancers with Rare Molecular Alterations; NCT04151342; n=3,879). Specifically, we reviewed data collected, study elements, protocol language, coordination, institutional review boards (IRBs), and contracts. We also performed stakeholder-directed survey and discussions, analysis of funding, research output, and collaborations, as well as a Strengths-Weaknesses-Opportunities-Threats (SWOT) analysis. Results: AUPROS is an innovative study design that borrows design elements from master protocol trials, adaptive trials, and master observational trials as well as retrospective and ambispective designs, enabling comprehensive data collection. The universality of AUPROS allowed for multi-purpose analyses of various real-world data (RWD) including epidemiological, clinical, patient-reported outcomes, biospecimens, and imaging data. The adaptive nature created opportunities for multi-source funding (e.g., academic, pharmaceutical, philanthropic, among others), as well as national/international collaborations and involvement in consortia (e.g., ILCCO, TCGA, BEACON, and INHANCE). Additionally, AUPROS has allowed the development of an expanding national RWE platform that involves collaborations among medical and radiation oncologists and surgeons across different tumor sites. As assessed by research output analyses, AUPROS provided a platform for enhanced research productivity in different areas and utilizing various data sources (METAL=130; THANKS=31; CARMA=5 publications). Our survey and SWOT analysis identified several cost and operational benefits of AUPROS over conventional observational studies, as well as several challenges pertinent to ethics approvals, sustainability, complex coordination, and data quality. Conclusions: AUPROS is an innovative model for real-world observational studies with significant logistical and methodological benefits over conventional RWD study designs. Our findings may help broaden the use of AUPROS to address emerging needs in the scope of precision oncology and clinico-epidemiological research. Citation Format: Samir H. Barghout, Stavroula Raptis, Luna Jia Zhan, Faisal Al-Agha, M Catherine Brown, Devalben Patel, Geoffrey Liu. Adaptive universal platform for real-world observational studies (AUPROS): An emerging model for clinical, epidemiologic, and precision oncology research [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 926.

Raising the Bar for Real-World Data in Oncology: Approaches to Quality Across Multiple Dimensions.

Electronic health record (EHR)-based real-world data (RWD) are integral to oncology research, and understanding fitness for use is critical for data users. Complexity of data sources and curation methods necessitate transparency into how quality is approached. We describe the application of data quality dimensions in curating EHR-derived oncology RWD. A targeted review was conducted to summarize data quality dimensions in frameworks published by the European Medicines Agency, The National Institute for Healthcare and Excellence, US Food and Drug Administration, Duke-Margolis Center for Health Policy, and Patient-Centered Outcomes Research Institute. We then characterized quality processes applied to curation of Flatiron Health RWD, which originate from EHRs of a nationwide network of academic and community cancer clinics, across the summarized quality dimensions. The primary quality dimensions across frameworks were relevance (including subdimensions of availability, sufficiency, and representativeness) and reliability (including subdimensions of accuracy, completeness, provenance, and timeliness). Flatiron Health RWD quality processes were aligned to each dimension. Relevancy to broad or specific use cases is optimized through data set size and variable breadth and depth. Accuracy is addressed using validation approaches, such as comparison with external or internal reference standards or indirect benchmarking, and verification checks for conformance, consistency, and plausibility, selected on the basis of feasibility and criticality of the variable to the intended use case. Completeness is assessed against expected source documentation; provenance by recording data transformation, management procedures, and auditable metadata; and timeliness by setting refresh frequency to minimize data lags. Development of high-quality, scaled, EHR-based RWD requires integration of systematic processes across the data lifecycle. Approaches to quality are optimized through knowledge of data sources, curation processes, and use case needs. By addressing quality dimensions from published frameworks, Flatiron Health RWD enable transparency in determining fitness for real-world evidence generation.

Use of Healthcare Claims Data to Generate Real-World Evidence on Patients With Drug-Resistant Epilepsy: Practical Considerations for Research.

Objectives: Regulatory bodies, health technology assessment agencies, payers, physicians, and other decision-makers increasingly recognize the importance of real-world evidence (RWE) to provide important and relevant insights on treatment patterns, burden/cost of illness, product safety, and long-term and comparative effectiveness. However, RWE generation requires a careful approach to ensure rigorous analysis and interpretation. There are limited examples of comprehensive methodology for the generation of RWE on patients who have undergone neuromodulation for drug-resistant epilepsy (DRE). This is likely due, at least in part, to the many challenges inherent in using real-world data to define DRE, neuromodulation (including type implanted), and related outcomes of interest. We sought to provide recommendations to enable generation of robust RWE that can increase knowledge of "real-world" patients with DRE and help inform the difficult decisions regarding treatment choices and reimbursement for this particularly vulnerable population. Methods: We drew upon our collective decades of experience in RWE generation and relevant disciplines (epidemiology, health economics, and biostatistics) to describe challenges inherent to this therapeutic area and to provide potential solutions thereto within healthcare claims databases. Several examples were provided from our experiences in DRE to further illustrate our recommendations for generation of robust RWE in this therapeutic area. Results: Our recommendations focus on considerations for the selection of an appropriate data source, development of a study timeline, exposure allotment (specifically, neuromodulation implantation for patients with DRE), and ascertainment of relevant outcomes. Conclusions: The need for RWE to inform healthcare decisions has never been greater and continues to grow in importance to regulators, payers, physicians, and other key stakeholders. However, as real-world data sources used to generate RWE are typically generated for reasons other than research, rigorous methodology is required to minimize bias and fully unlock their value.

Use of Healthcare Claims Data to Generate Real-World Evidence on Patients With Drug-Resistant Epilepsy: Practical Considerations for Research

Objectives: Regulatory bodies, health technology assessment agencies, payers, physicians, and other decision-makers increasingly recognize the importance of real-world evidence (RWE) to provide important and relevant insights on treatment patterns, burden/cost of illness, product safety, and long-term and comparative effectiveness. However, RWE generation requires a careful approach to ensure rigorous analysis and interpretation. There are limited examples of comprehensive methodology for the generation of RWE on patients who have undergone neuromodulation for drug-resistant epilepsy (DRE). This is likely due, at least in part, to the many challenges inherent in using real-world data to define DRE, neuromodulation (including type implanted), and related outcomes of interest. We sought to provide recommendations to enable generation of robust RWE that can increase knowledge of “real-world” patients with DRE and help inform the difficult decisions regarding treatment choices and reimbursement for this particularly vulnerable population. Methods: We drew upon our collective decades of experience in RWE generation and relevant disciplines (epidemiology, health economics, and biostatistics) to describe challenges inherent to this therapeutic area and to provide potential solutions thereto within healthcare claims databases. Several examples were provided from our experiences in DRE to further illustrate our recommendations for generation of robust RWE in this therapeutic area. Results: Our recommendations focus on considerations for the selection of an appropriate data source, development of a study timeline, exposure allotment (specifically, neuromodulation implantation for patients with DRE), and ascertainment of relevant outcomes. Conclusions: The need for RWE to inform healthcare decisions has never been greater and continues to grow in importance to regulators, payers, physicians, and other key stakeholders. However, as real-world data sources used to generate RWE are typically generated for reasons other than research, rigorous methodology is required to minimize bias and fully unlock their value.