Augmenting clinical trial data with external controls through energy balancing weighted power prior

Background: The higher cost of confirmatory trials and ethical considerations for patients with the existing treatment options led to the investigation of new trial designs. There have been multiple efforts to develop statistical methodologies to evaluate the impact of borrowing external and historical randomized clinical trials (RCT) data sources in a hybrid control design setting. However, few studies used real-world data and historical RCT while comparing Bayesian and frequentist approaches under similar conditions. We evaluated Bayesian (commensurate prior model) and frequentist (Frailty Cox model) hierarchical methods in hybrid control simulation. In addition, dynamic borrowing between the control arms of two historical RCTs in previously treated non-small cell lung cancer (NSCLC) was tested. Methods: Bayesian and frequentist models were carried out and evaluated through computational simulations to mimic 300 repeated clinical trial experiments. Estimation of bias, power, and type I error was conducted. The critical value for testing of 5% (two-sided type I error) was used. Half-Cauchy hyperprior was chosen for the Bayesian commensurate approach. Normal prior for heterogeneity between RCT and the external data was used for Bayesian commensurate prior and Frailty Cox model. Results: Model performance, in terms of estimator bias, power, and Type I error, was similar between Bayesian and frequentist approaches. Estimator bias of the two models was within 0.07 (Log(HR), |β^-β|). Type I errors estimated by the two methods ranged from 3% and 10%. Slight type I error inflation was observed when there were differences between data sources. Power to detect treatment effect using various scenarios ranged from 60% to 85%. Power gain was demonstrated with sufficient commensurability between RCT and external control (81% to 82% compared to 75% without borrowing) and potential power loss observed with more differences between data sources (58%-66% compared to 75% without borrowing). When applying this methodology with historical RCTs, results were consistent with simulation scenarios. Conclusions: We demonstrated similar model performance for Bayesian and Frailty Cox models in estimating treatment effects in hybrid control simulation and existing RCT. While some differences in models’ setup, availability of standard software, and flexibility exists, a successful construct of hybrid control can lead to accelerated internal decision making and streamlined clinical trial design. However, many factors can affect treatment effect estimates and careful consideration of the differences between data sources, distribution of estimates, and direction of the bias has to be taken before implementing this approach in a clinical trial setting. Citation Format: Fu-Chi Hsieh, Natalia Sadetsky, Xiao Li, Ruilin Li, Jiawen Zhu. Real-world data in hybrid clinical trial design: Considerations and impact on treatment effect [abstract]. In: Proceedings of the AACR Special Conference on Advancing Precision Medicine Drug Development: Incorporation of Real-World Data and Other Novel Strategies; Jan 9-12, 2020; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2020;26(12_Suppl_1):Abstract nr 05.

Integrating external data into a clinical trial can introduce systematic bias in estimates and inflate the study's type I error due to differences in study design and enrollment criteria. Existing prior designs for information borrowing lack the ability to dynamically adjust the weight based on the similarity between concurrent and external data. To address this challenge, we thereby introduce a novel method called the elastic commensurate prior (ECP), which combines the commensurate prior with the elastic prior method. By dynamically adjusting the weight of external data using a measure of congruence, this method demonstrates strong performance in maintaining power while providing adequate type I error control across different scenarios, including congruence, approximate congruence, and incongruence between external and concurrent data. Compared to existing methods such as the modified power prior, meta-analytic-predictive (MAP) prior, robust MAP prior, non-informative prior, and fully informative prior, the ECP method is flexible and performs well across all settings. Furthermore, our method also allows for the integration of covariates in estimating data congruence for dynamic information borrowing, achieving both strong performance in power and adequate control of type I error. Overall, the ECP represents a promising option for leveraging external data in clinical trials, reducing costs by decreasing the sample size requirement, and thereby accelerating research and drug development timelines.

Background: Reference to so-called real-world data is more often made in marketing authorization applications for medicines intended to diagnose, prevent or treat rare diseases compared to more common diseases. We provide granularity on the type and aim of any external data on efficacy aspects from both real-world data sources and external trial data as discussed in regulatory submissions of orphan designated medicinal products in the EU. By quantifying the contribution of external data according to various regulatory characteristics, we aimed at identifying specific opportunities for external data in the field of orphan conditions.Methods: Information on external data in regulatory documents covering 72 orphan designations was extracted. Our sample comprised public assessment reports for approved, refused, or withdrawn applications concluded from 2019–2021 at the European Medicines Agency. Products with an active orphan designation at the time of submission were scrutinized regarding the role of external data on efficacy aspects in the context of marketing authorization applications, or on the criterion of “significant benefit” for the confirmation of the orphan designation at the time of licensing. The reports allowed a broad distinction between clinical development, regulatory decision making, and intended post-approval data collection. We defined three categories of external data, administrative data, structured clinical data, and external trial data (from clinical trials not sponsored by the applicant), and noted whether external data concerned the therapeutic context of the disease or the product under review.Results: While reference to external data with respect to efficacy aspects was included in 63% of the approved medicinal products in the field of rare diseases, 37% of marketing authorization applications were exclusively based on the dedicated clinical development plan for the product under review. Purely administrative data did not play any role in our sample of reports, but clinical data collected in a structured manner (from routine care or clinical research) were often used to inform on the trial design. Two additional recurrent themes for the use of external data were the contextualization of results, especially to confirm the orphan designation at the time of licensing, and reassurance of a large difference in treatment effect size or consistency of effects observed in clinical trials and practice. External data on the product under review were restricted to either active substances already belonging to the standard of care even before authorization or to compassionate use schemes. Furthermore, external data were considered pivotal for marketing authorization only exceptionally and only for active substances already in use within the specific therapeutic indication. Applications for the rarest conditions and those without authorized treatment alternatives were especially prominent with respect to the use of external data from real-world data sources both in the pre- and post-approval setting.Conclusion: Specific opportunities for external data in the setting of marketing authorizations in the field of rare diseases were identified. Ongoing initiatives of fostering systematic data collection are promising steps for a more efficient medicinal product development in the field of rare diseases.

Current perspectives for external control arms in oncology clinical trials: Analysis of EMA approvals 2016–2021

The use of real-world data (RWD) external control arms in prospective studies is increasing. The advantages, including the immediate availability of a control population, must be balanced with the requirements of meeting evidentiary standards. To address the question of whether and to what extent the methods of RWD studies compare to standard methods used in randomized clinical trials. A systematic search across 4 electronic databases and Google Scholar was conducted from January 1, 2000, to October 23, 2023. Studies were included in the systematic review if they compared an intervention arm in a clinical trial to an RWD control arm in patients with hematological cancers and if they were published between 2000 and 2023. Thirty-two prospective intervention studies incorporating external control data from RWD sources of patients with hematological cancers were identified. A total of 4306 patients from intervention arms and 10 594 from RWD control arms were included across all studies. Only 2 studies (6%) included prospectively collected RWD. The complete trial inclusion criteria were applied to the RWD cohort in 7 studies (22%). Four studies (13%) published the statistical analysis plan and prespecified use of RWD. A total of 23 studies (72%) applied matching algorithms for trial and RWD cohorts, including matching for demographic, disease, and/or therapy-related characteristics. The end point criteria were the same as the trial in 8 studies (25%). In contrast, 12 studies (38%) used different end points, and 12 (38%) did not provide an end point definition for the RWD. Twelve studies (38%) had a median follow-up difference of less than a year between arms. Eight studies (25%) reported toxic effect data for the trial arm, of which 5 studies reported toxic effect data for the RWD arm. In this systematic review, limitations were observed in the application of clinical trial eligibility criteria to RWD, statistical rigor and application of matching methods, the definition of end points, follow-up, and reporting of adverse events, which may challenge the conclusions reported in studies using RWD.

Randomized controlled trials (RCT) have been the gold standard for evaluating medical treatments for many decades. Randomization reduces systematic biases resulting from treatment or patient selection, whereby the improvement in clinical outcomes may be attributed to the experimental therapy under study. However, RCTs are often criticized for requiring large sample sizes and taking a long time to complete. For newly diagnosed glioblastoma (GBM), the clinical trial landscape has seen little progress since the establishment of the standard of care (SOC) by Stupp. Given the urgent need for better therapies, it has been argued that data collected from patients treated with the SOC from past GBM trials can provide high-quality external control data to supplement concurrent control arm in future trials, thereby increasing drug development efficiency by reducing the number of patients treated with SOC. Herein we consider a new design approach that leverages historical control data in the design and analysis of phase 3 GBM trials. At the first stage, patients are randomized with an equal probability to standard (concurrent control) arm and experimental arm. An interim analysis entails an outcome comparison between the concurrent and external control arms. If comparability is established, the external control data are carried forward to be combined with concurrent control data at the second stage where the randomization ratio is adapted to favor the experimental therapy, thereby reducing the number of patients treated in the concurrent control arm. Using completed phase 3 GBM trials, we elucidate the potential gain in design efficiency and draw caution to scenarios where it may fall short on meeting statistical criteria. We highlight practical challenges in its implementation and conclude that the new method is not ready for definitive phase 3 GBM studies at the current time. This work represents a critical appraisal of this new concept in GBM.

Suppose we are interested in the effect of a treatment in a clinical trial. The efficiency of inference may be limited due to small sample size. However, external control data are often available from historical studies. Motivated by an application to Helicobacter pylori infection, we show how to borrow strength from such data to improve efficiency of inference in the clinical trial. Under an exchangeability assumption about the potential outcome mean, we show that the semiparametric efficiency bound for estimating the average treatment effect can be reduced by incorporating both the clinical trial data and external controls. We then derive a doubly robust and locally efficient estimator. The improvement in efficiency is prominent especially when the external control data set has a large sample size and small variability. Our method allows for a relaxed overlap assumption, and we illustrate with the case where the clinical trial only contains a treated group. We also develop doubly robust and locally efficient approaches that extrapolate the causal effect in the clinical trial to the external population and the overall population. Our results also offer a meaningful implication for trial design and data collection. We evaluate the finite-sample performance of the proposed estimators via simulation. In the Helicobacter pylori infection application, our approach shows that the combination treatment has potential efficacy advantages over the tripletherapy.

BackgroundPlatform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.MethodsWe conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.ResultsOnly 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.ConclusionsStatistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

Externally controlled trials (ECTs) can serve as an alternative in settings where randomized clinical trials (RCTs) are unfeasible. However, the methodological rigor of ECTs, particularly with regard to bias control, is often inadequately assessed, which can compromise the validity of studies and lead to incorrect decisions. To examine the design, conduct, and analysis characteristics of current ECTs and to assess whether appropriate methods were used to control bias. This cross-sectional study searched PubMed for ECTs published between January 1, 2010, and December 31, 2023. Eligible ECTs included single-arm trials with an external control or that used a treatment group from an RCT compared with an external control and evaluated the efficacy and/or safety of a drug or medical device. Data analysis was conducted from March 5 to 20, 2025. Extracted information included design characteristics, external control data sources, transparency in covariate selection, statistical methods, and the use of sensitivity and quantitative bias analyses. The characteristics of included ECTs were compared between journals in the top 25% in their Journal Citation Reports category (Q1) and non-Q1. This study included 180 ECTs, of which 85 (47.2%) focused on oncology. Only 64 (35.6%) provided reasons for using external controls, and 29 (16.1%) were prespecified to use external controls. The main sources of external controls were clinical (also termed real-world) data (98 [54.4%]) and trial-derived controls (67 [37.2%]), while concurrent data collection with the treatment arm was relatively infrequent (18 [10.0%]). Only 14 studies (7.8%) conducted feasibility assessments to evaluate the adequacy of data sources, and 13 (7.2%) specified how to handle missing data in external control datasets. Covariate selection procedures were described in 37 of the 164 studies (22.6%) that reported important covariates. Sixty studies (33.3%) used statistical methods to adjust for important covariates when generating the external control, with the propensity score method being the most common (35 of 60 [58.3%]). Among 120 ECTs that generated external controls without statistical methods, 91 (75.8%) used univariate analysis to estimate treatment effects, and only 18 (15.0%) used multivariable regression analysis. Sensitivity analyses for primary outcomes were performed in 32 studies (17.8%), and quantitative bias analyses (2 [1.1%]) were nearly absent. ECTs in Q1 journals were more likely to prespecify the use of external controls (χ21 = 9.86; P = .002) and provided rationales for using external controls (χ21 = 4.33; P = .04). Thirteen recommendations for the careful practice of ECTs are proposed. In this cross-sectional study of ECTs, current practices in the design, conduct, and analysis were suboptimal, limiting their reliability and credibility. The study identified several critical methodological issues, such as the lack of justification for using external controls, failure to prespecify external controls in the protocol, insufficient use of confounding adjustment techniques, inadequate sensitivity analyses, and almost complete absence of quantitative bias analyses. Therefore, actionable suggestions for future ECT practices are proposed.

Results from single-arm clinical trials can be contextualized by comparing against external controls (ECs) derived from real-world data (RWD). However, lack of randomization and differences in variable capture between data sources may introduce bias into estimates of treatment effect and standard error, the extent of which can be assessed via meta-analysis of comparisons between clinical trial control arms and their EC replicates. Clinical trial progression-free survival (PFS) outcomes from the 14 chemotherapy control arms of 12 non-small cell lung cancer clinical trials were replicated using the US nationwide deidentified Flatiron Health electronic health record-derived database, with real-world PFS (rwPFS) as the end point. A meta-analysis of loge hazard ratios (HRs) comparing randomized controlled trial (RCT) and RWD control arms was conducted. For illustration, the meta-analysis results were used to restore correct operating characteristics of a hypothetical prospective single-arm study with EC. With the exception of one outlier, rwPFS outcomes were on average similar to PFS outcomes, albeit with substantial between-study variation. RCT compared with RWD arms differed by a mean loge HR of -0.001, with a standard deviation of 0.164 (including the outlier). Applying these estimates to adjust error probabilities in a hypothetical prospective EC study revealed that between-study variation of bias in this setting should be adjusted for, to avoid incorrect decision making. The close alignment of results between RCT and RWD increases confidence that RWD ECs using the rwPFS end point in this disease setting can provide context for future single-arm clinical trials despite potential differences in end point assessment.

This guidance provides recommendations to sponsors and investigators considering the use of externally controlled clinical trials to provide evidence of the safety and effectiveness of a drug product. In an externally controlled trial, outcomes in participants receiving the test treatment according to a protocol are compared with outcomes in a group of people external to the trial who had not received the same treatment. The external control arm can be a group of people, treated or untreated, from an earlier time (historical control), or it can be a group of people, treated or untreated, during the same period (concurrent control) but in another setting. The guidance addresses considerations for the design and analysis of externally controlled trials to study the effectiveness and safety of drugs, including discussion of threats to the validity of trial results from potential bias. Although various sources of data can serve as the control arm in an externally controlled trial, this guidance focuses on the use of patient-level data from other clinical trials or from real-world data (RWD) sources, such as registries, electronic health records, and medical claims. The guidance also describes considerations related to communicating with the FDA and ensuring access by the FDA to data from an externally controlled trial. This guidance does not address other types of external controls, such as using summary-level estimates instead of patient-level data. This guidance does not discuss details of the design and analysis of a natural history study nor the reliability and relevance of various sources of RWD that could be used in an externally controlled trial. Finally, this guidance also does not discuss considerations for using external control data to supplement a control arm in a traditional randomized controlled clinical trial. In general, FDA guidance documents do not establish legally enforceable responsibilities. Instead, guidance describes the Agency’s current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidance means that something is suggested or recommended, but not required.

External control (EC) data from completed clinical trials and electronic health records can be valuable for the design and analysis of future clinical trials. We discuss the use of EC data for early stopping decisions in randomized clinical trials (RCTs). We specify interim analyses (IAs) approaches for RCTs, which allow investigators to integrate external data into early futility stopping decisions. IAs utilize predictions based on early data from the RCT, possibly combined with external data. These predictions at IAs express the probability that the trial will generate significant evidence of positive treatment effects. The trial is discontinued if this predictive probability becomes smaller than a prespecified threshold. We quantify efficiency gains and risks associated with the integration of external data into interim decisions. We then analyze a collection of glioblastoma (GBM) data sets, to investigate if the balance of efficiency gains and risks justify the integration of external data into the IAs of future GBM RCTs. Our analyses illustrate the importance of accounting for potential differences between the distributions of prognostic variables in the RCT and in the external data to effectively leverage external data for interim decisions. Using GBM data sets, we estimate that the integration of external data increases the probability of early stopping of ineffective experimental treatments by up to 25% compared to IAs that do not leverage external data. Additionally, we observe a reduction of the probability of early discontinuation for effective experimental treatments, which improves the RCT power. Leveraging external data for IAs in RCTs can support early stopping decisions and reduce the number of enrolled patients when the experimental treatment is ineffective.

Augmented randomized clinical trials are a valuable design option for early phase clinical trials. The addition of external controls could, on the one hand, increase precision in treatment effect estimates or reduce the number of required control patients for a randomized trial but may, on the other hand, introduce bias. We build on previous work on augmented trials with one external control data source in time-to-event settings and extend it to multiple control data sources. In a comprehensive simulation study, we evaluate existing and novel analysis options mainly based on Bayesian hierarchical models as well as propensity score analysis. Different sources of bias are investigated including population (observable and unobservable confounders), data collection (assessment schedule, real-world vs. clinical trial data), and time trend as well as different types of data like individual patient data (with or without baseline covariates) or summary data. Our simulation study provides recommendations in terms of choice of estimation method as well as choice of data sources. Explicit incorporation of the above-mentioned sources of bias in a simulation study is relevant as the magnitude of deviation from the ideal setting has a significant impact on all investigated estimation methods.

<div>AbstractPurpose:<p>We discuss designs and interpretable metrics of bias and statistical efficiency of “externally controlled” trials (ECT) and compare ECT performance to randomized and single-arm designs.</p>Experimental Design:<p>We specify an ECT design that leverages information from real-world data (RWD) and prior clinical trials to reduce bias associated with interstudy variations of the enrolled populations. We then used a collection of clinical studies in glioblastoma (GBM) and RWD from patients treated with the current standard of care to evaluate ECTs. Validation is based on a “leave one out” scheme, with iterative selection of a single-arm from one of the studies, for which we estimate treatment effects using the remaining studies as external control. This produces interpretable and robust estimates on ECT bias and type I errors.</p>Results:<p>We developed a model-free approach to evaluate ECTs based on collections of clinical trials and RWD. For GBM, we verified that inflated false positive error rates of standard single-arm trials can be considerably reduced (up to 30%) by using external control data.</p>Conclusions:<p>The use of ECT designs in GBM, with adjustments for the clinical profiles of the enrolled patients, should be preferred to single-arm studies with fixed efficacy thresholds extracted from published results on the current standard of care.</p></div>

We discuss designs and interpretable metrics of bias and statistical efficiency of "externally controlled" trials (ECT) and compare ECT performance to randomized and single-arm designs. We specify an ECT design that leverages information from real-world data (RWD) and prior clinical trials to reduce bias associated with interstudy variations of the enrolled populations. We then used a collection of clinical studies in glioblastoma (GBM) and RWD from patients treated with the current standard of care to evaluate ECTs. Validation is based on a "leave one out" scheme, with iterative selection of a single-arm from one of the studies, for which we estimate treatment effects using the remaining studies as external control. This produces interpretable and robust estimates on ECT bias and type I errors. We developed a model-free approach to evaluate ECTs based on collections of clinical trials and RWD. For GBM, we verified that inflated false positive error rates of standard single-arm trials can be considerably reduced (up to 30%) by using external control data. The use of ECT designs in GBM, with adjustments for the clinical profiles of the enrolled patients, should be preferred to single-arm studies with fixed efficacy thresholds extracted from published results on the current standard of care.