Evaluation Of Surrogate Endpoints Research Articles

Comparing Bayesian hierarchical meta-regression methods and evaluating the influence of priors for evaluations of surrogate endpoints on heterogeneous collections of clinical trials

BackgroundSurrogate endpoints, such as those of interest in chronic kidney disease (CKD), are often evaluated using Bayesian meta-regression. Trials used for the analysis can evaluate a variety of interventions for different sub-classifications of disease, which can introduce two additional goals in the analysis. The first is to infer the quality of the surrogate within specific trial subgroups defined by disease or intervention classes. The second is to generate more targeted subgroup-specific predictions of treatment effects on the clinical endpoint.MethodsUsing real data from a collection of CKD trials and a simulation study, we contrasted surrogate endpoint evaluations under different hierarchical Bayesian approaches. Each approach we considered induces different assumptions regarding the relatedness (exchangeability) of trials within and between subgroups. These include partial-pooling approaches, which allow subgroup-specific meta-regressions and, yet, facilitate data adaptive information sharing across subgroups to potentially improve inferential precision. Because partial-pooling models come with additional parameters relative to a standard approach assuming one meta-regression for the entire set of studies, we performed analyses to understand the impact of the parameterization and priors with the overall goals of comparing precision in estimates of subgroup-specific meta-regression parameters and predictive performance.ResultsIn the analyses considered, partial-pooling approaches to surrogate endpoint evaluation improved accuracy of estimation of subgroup-specific meta-regression parameters relative to fitting separate models within subgroups. A random rather than fixed effects approach led to reduced bias in estimation of meta-regression parameters and in prediction in subgroups where the surrogate was strong. Finally, we found that subgroup-specific meta-regression posteriors were robust to use of constrained priors under the partial-pooling approach, and that use of constrained priors could facilitate more precise prediction for clinical effects in trials of a subgroup not available for the initial surrogacy evaluation.ConclusionPartial-pooling modeling strategies should be considered for surrogate endpoint evaluation on collections of heterogeneous studies. Fitting these models comes with additional complexity related to choosing priors. Constrained priors should be considered when using partial-pooling models when the goal is to predict the treatment effect on the clinical endpoint.

SA13 Can Models for Surrogate Endpoint Evaluation Be Used to Predict Generic Measures From Disease Specific Measures of Health Related Quality of Life Based on Summary Data?

SA13 Can Models for Surrogate Endpoint Evaluation Be Used to Predict Generic Measures From Disease Specific Measures of Health Related Quality of Life Based on Summary Data?

1102P Evaluation of surrogate endpoints for overall survival within the RELATIVITY-047 trial

1102P Evaluation of surrogate endpoints for overall survival within the RELATIVITY-047 trial

Handling missing within-study correlations in the evaluation of surrogate endpoints.

Rigorous evaluation of surrogate endpoints is performed in a trial-level analysis in which the strength of the association between treatment effects on the clinical and surrogate endpoints is quantified across a collection of previously conducted trials. To reduce bias in measures of the performance of the surrogate, the statistical model must account for the sampling error in each trial's estimated treatment effects and their potential correlation. Unfortunately, these within-study correlations can be difficult to obtain, especially for meta-analysis of published trial results where individual patient data is not available. As such, these terms are frequently partially or completely missing in the analysis. We show that improper handling of these missing terms can meaningfully alter the perceived quality of the surrogate and we introduce novel strategies to handle the missingness.

P49 Evaluation of Surrogate Endpoints (SES) for Previously Untreated Unresectable or Metastatic Melanoma (MMEL): Analyses from a Longitudinal Electronic Health Record Database in the United States (US)

P49 Evaluation of Surrogate Endpoints (SES) for Previously Untreated Unresectable or Metastatic Melanoma (MMEL): Analyses from a Longitudinal Electronic Health Record Database in the United States (US)

CO161 Evaluation of Surrogate Endpoints for Overall Survival in Platinum-Resistant/Refractory Ovarian Cancer

Recent publications did not find strong evidence to support the surrogacy of progression-free survival (PFS) for overall survival (OS) in patients with ovarian cancer patients undergoing first-line platinum-based therapy. The surrogacy analysis could be confounded by multiple factors, including post-relapse treatments. To investigate this topic further we assessed whether PFS may be a surrogate for OS in patients with platinum-resistant/refractory ovarian cancer (PROC).

Using Bayesian Evidence Synthesis Methods to Incorporate Real-World Evidence in Surrogate Endpoint Evaluation.

Traditionally, validation of surrogate endpoints has been carried out using randomized controlled trial (RCT) data. However, RCT data may be too limited to validate surrogate endpoints. In this article, we sought to improve the validation of surrogate endpoints with the inclusion of real-world evidence (RWE). We use data from comparative RWE (cRWE) and single-arm RWE (sRWE) to supplement RCT evidence for the evaluation of progression-free survival (PFS) as a surrogate endpoint to overall survival (OS) in metastatic colorectal cancer (mCRC). Treatment effect estimates from RCTs, cRWE, and matched sRWE, comparing antiangiogenic treatments with chemotherapy, were used to inform surrogacy patterns and predictions of the treatment effect on OS from the treatment effect on PFS. Seven RCTs, 4 cRWE studies, and 2 matched sRWE studies were identified. The addition of RWE to RCTs reduced the uncertainty around the estimates of the parameters for the surrogate relationship. The addition of RWE to RCTs also improved the accuracy and precision of predictions of the treatment effect on OS obtained using data on the observed effect on PFS. The addition of RWE to RCT data improved the precision of the parameters describing the surrogate relationship between treatment effects on PFS and OS and the predicted clinical benefit of antiangiogenic therapies in mCRC. Regulatory agencies increasingly rely on surrogate endpoints when making licensing decisions, and for the decisions to be robust, surrogate endpoints need to be validated. In the era of precision medicine, when surrogacy patterns may depend on the drug's mechanism of action and trials of targeted therapies may be small, data from randomized controlled trials may be limited.Real-world evidence (RWE) is increasingly used at different stages of the drug development process. When used to enhance the evidence base for surrogate endpoint evaluation, RWE can improve inferences about the strength of surrogate relationships and the precision of predicted treatment effect on the final clinical outcome based on the observed effect on the surrogate endpoint in a new trial.Careful selection of RWE is needed to reduce risk of bias.

Development of Microbiome Biomarkers in Intervention Studies.

There has been an increased interest in studying the association between microbial communities and different diseases and in discovering microbiome biomarkers. This association is pivotal to discover such biomarkers. In this paper, we present a unified modelling approach that can be used to detect and develop microbiome biomarkers for different clinical responses of interest at different levels of the microbiome ecosystem. We extended the methodology rooted in the information theory and joint modelling approaches for the evaluation of surrogate endpoints in randomized clinical trials to the high-dimensional microbiome setting. The unified modelling approach introduced in this paper allows for detecting biomarkers associated with a clinical response of interest, adjusting for the intervention applied to the subjects. For some microbiome features, the association is driven by the treatment, while for others, the association reflects the correlation between the microbiome biomarker and the clinical response of interest. The results have demonstrated that biomarkers can be identified at different levels of the microbiome phylogenetic tree using various measures as biomarkers.

P42 Using Bayesian Evidence Synthesis Methods to Incorporate Real-World Evidence in Surrogate Endpoint Evaluation

P42 Using Bayesian Evidence Synthesis Methods to Incorporate Real-World Evidence in Surrogate Endpoint Evaluation

RWD45 Real-World Data to Support Surrogate Endpoint Evaluation for Health Technology Assessment

To identify levels of validity of a surrogate endpoint (SE), hurdles to their acceptance and how Real World Data (RWD) can support SE evaluation in the context of Health Technology Assessment (HTA).

The Validity of Surrogate Endpoints in Sub Groups of Metastatic Colorectal Cancer Patients Defined by Treatment Class and KRAS Status.

Background and Aim: Findings from the literature suggest that the validity of surrogate endpoints in metastatic colorectal cancer (mCRC) may depend on a treatments' mechanism of action. We explore this and the impact of Kirsten rat sarcoma (KRAS) status on surrogacy patterns in mCRC. Methods: A systematic review was undertaken to identify randomized controlled trials (RCTs) for pharmacological therapies in mCRC. Bayesian meta-analytic methods for surrogate endpoint evaluation were used to evaluate surrogate relationships across all RCTs, by KRAS status and treatment class. Surrogate endpoints explored were progression free survival (PFS) as a surrogate endpoint for overall survival (OS), and tumour response (TR) as a surrogate for PFS and OS. Results: 66 RCTs were identified from the systematic review. PFS showed a strong surrogate relationship with OS across all data and in subgroups by KRAS status. The relationship appeared stronger within individual treatment classes compared to the overall analysis. The TR-PFS and TR-OS relationships were found to be weak overall but stronger within the Epidermal Growth Factor Receptor + Chemotherapy (EGFR + Chemo) treatment class; both overall and in the wild type (WT) patients for TR-PFS, but not in patients with the mutant (MT) KRAS status where data were limited. Conclusions: PFS appeared to be a good surrogate endpoint for OS. TR showed a moderate surrogate relationship with PFS and OS for the EGFR + Chemo treatment class. There was some evidence of impact of the mechanism of action on the strength of the surrogacy patterns in mCRC, but little evidence of the impact of KRAS status on the validity of surrogate endpoints.

Use of copula to model within-study association in bivariate meta-analysis of binomial data at the aggregate level: ABayesian approach and application to surrogate endpoint evaluation.

Bivariate meta-analysis provides a useful framework for combining information across related studies and has been utilized to combine evidence from clinical studies to evaluate treatment efficacy on two outcomes. It has also been used to investigate surrogacy patterns between treatment effects on the surrogate endpoint and the final outcome. Surrogate endpoints play an important role in drug development when they can be used to measure treatment effect early compared to the final outcome and to predict clinical benefit or harm. The standard bivariate meta-analytic approach models the observed treatment effects on the surrogate and the final outcome outcomes jointly, at both the within-study and between-studies levels, using a bivariate normal distribution. For binomial data, a normal approximation on log odds ratio scale can be used. However, this method may lead to biased results when the proportions of events are close to one or zero, affecting the validation of surrogate endpoints. In this article, we explore modeling the two outcomes on the original binomial scale. First, we present a method that uses independent binomial likelihoods to model the within-study variability avoiding to approximate the observed treatment effects. However, the method ignores the within-study association. To overcome this issue, we propose a method using a bivariate copula with binomial marginals, which allows the model to account for the within-study association. We applied the methods to an illustrative example in chronic myeloid leukemia to investigate the surrogate relationship between complete cytogenetic response and event-free-survival.

Evaluation of Surrogate Endpoints Using Information-Theoretic Measure of Association Based on Havrda and Charvat Entropy.

Surrogate endpoints have been used to assess the efficacy of a treatment and can potentially reduce the duration and/or number of required patients for clinical trials. Using information theory, Alonso et al. (2007) proposed a unified framework based on Shannon entropy, a new definition of surrogacy that departed from the hypothesis testing framework. In this paper, a new family of surrogacy measures under Havrda and Charvat (H-C) entropy is derived which contains Alonso’s definition as a particular case. Furthermore, we extend our approach to a new model based on the information-theoretic measure of association for a longitudinally collected continuous surrogate endpoint for a binary clinical endpoint of a clinical trial using H-C entropy. The new model is illustrated through the analysis of data from a completed clinical trial. It demonstrates advantages of H-C entropy-based surrogacy measures in the evaluation of scheduling longitudinal biomarker visits for a phase 2 randomized controlled clinical trial for treatment of multiple sclerosis.

POSB285 Five Identified Hurdles Associated with Surrogate Endpoint Validation

Surrogate endpoints are adopted in clinical trials when final clinical endpoints are difficult or costly to be measured within the timeframe of clinical studies. The suggestion to establish the correlation between surrogate and final endpoints was proposed in 1996 and a conceptual approach based on the recommendation was published. In 2008, the US FDA set up Biomarker Qualification Program to validate surrogate endpoints. Health technology assessment bodies also released principles on surrogate endpoint evaluation.

Implementing the meta-analytic approach for the evaluation of surrogate endpoints in SAS and R: a word of caution

ABSTRACT The meta-analytic approach has become the gold-standard methodology for the evaluation of surrogate endpoints and several implementations are currently available in SAS and R. The methodology is based on hierarchical models that are numerically demanding and, when the amount of data is limited, maximum likelihood algorithms may not converge or may converge to an ill-conditioned maximum such as a boundary solution. This may produce misleading conclusions and have negative implications for the evaluation of new drugs. In the present work, we explore the use of two distinct functions in R (lme and lmer) and the MIXED procedure in SAS to assess the validity of putative surrogate endpoints in the meta-analytic framework, via simulations and the analysis of a real case study. We describe some problems found with the lmer function in R that led to a poorer performance as compared with the lme function and MIXED procedure.

Surrogate endpoint evaluation using data from one large global randomized controlled trial

BackgroundRobust identification of surrogate endpoints can help accelerate the development of pharmacotherapies for diseases traditionally evaluated using true endpoints associated with prolonged follow-up. The meta-analysis-based surrogate endpoint evaluation (SEE) integrates data from multiple, usually smaller, trials to statistically confirm a surrogate endpoint as a robust proxy for the true endpoint. To test the applicability of SEE when only a single, larger trial is available, we analysed the cardiovascular (CV) survival endpoint from the large multinational trial LEADER (9340 subjects) that confirmed the CV safety of a diabetes drug (liraglutide). We evaluated if using country as a trial unit adequately facilitated the meta-analysis and calculation of R2 by country group.MethodsData were grouped by country, ensuring at least 30 CV deaths (497 in total) in each of the nine resulting by-country groups. In a two-step SEE on the grouped dataset, we first fitted the group-specific Cox proportional hazard models; next, on the trial-level, we regressed the estimated hazard ratio (HR; liraglutide vs placebo) of the true endpoints (CV death: 497 events, or all-cause death: 828 events) on the HR of the surrogate endpoint (major CV adverse event [MACE]: 1302 events) and derived the group-specific R2 and its 95% confidence interval (CI).ResultsGroup-level surrogacy of MACE was supported for CV death but not for all-cause death, with {text{R}}_{{{text{group}}}}^{2} values of 0.85 [0.63;1.00]95% CI and 0.23 [0.00;0.67]95% CI, respectively. Sensitivity analyses using different grouping approaches (e.g. grouping by region) corroborated the robustness of the conclusions as well as the appropriateness of the data-grouping approaches.ConclusionsWe derived a specific grouping approach to successfully apply SEE on data from a single trial. This may allow for the statistically robust identification and validation of surrogate endpoints based on the abundance of large monolithic outcome trials conducted as part of drug development programmes in, for example, diabetes.

Bayesian hierarchical meta-analytic methods for modeling surrogate relationships that vary across treatment classes using aggregate data.

Surrogate endpoints play an important role in drug development when they can be used to measure treatment effect early compared to the final clinical outcome and to predict clinical benefit or harm. Such endpoints are assessed for their predictive value of clinical benefit by investigating the surrogate relationship between treatment effects on the surrogate and final outcomes using meta‐analytic methods. When surrogate relationships vary across treatment classes, such validation may fail due to limited data within each treatment class. In this paper, two alternative Bayesian meta‐analytic methods are introduced which allow for borrowing of information from other treatment classes when exploring the surrogacy in a particular class. The first approach extends a standard model for the evaluation of surrogate endpoints to a hierarchical meta‐analysis model assuming full exchangeability of surrogate relationships across all the treatment classes, thus facilitating borrowing of information across the classes. The second method is able to relax this assumption by allowing for partial exchangeability of surrogate relationships across treatment classes to avoid excessive borrowing of information from distinctly different classes. We carried out a simulation study to assess the proposed methods in nine data scenarios and compared them with subgroup analysis using the standard model within each treatment class. We also applied the methods to an illustrative example in colorectal cancer which led to obtaining the parameters describing the surrogate relationships with higher precision.

Bivariate network meta-analysis for surrogate endpoint evaluation.

Surrogate endpoints are very important in regulatory decision making in healthcare, in particular if they can be measured early compared to the long‐term final clinical outcome and act as good predictors of clinical benefit. Bivariate meta‐analysis methods can be used to evaluate surrogate endpoints and to predict the treatment effect on the final outcome from the treatment effect measured on a surrogate endpoint. However, candidate surrogate endpoints are often imperfect, and the level of association between the treatment effects on the surrogate and final outcomes may vary between treatments. This imposes a limitation on methods which do not differentiate between the treatments. We develop bivariate network meta‐analysis (bvNMA) methods, which combine data on treatment effects on the surrogate and final outcomes, from trials investigating multiple treatment contrasts. The bvNMA methods estimate the effects on both outcomes for all treatment contrasts individually in a single analysis. At the same time, they allow us to model the trial‐level surrogacy patterns within each treatment contrast and treatment‐level surrogacy, thus enabling predictions of the treatment effect on the final outcome either for a new study in a new population or for a new treatment. Modelling assumptions about the between‐studies heterogeneity and the network consistency, and their impact on predictions, are investigated using an illustrative example in advanced colorectal cancer and in a simulation study. When the strength of the surrogate relationships varies across treatment contrasts, bvNMA has the advantage of identifying treatment comparisons for which surrogacy holds, thus leading to better predictions.

A reflection on the causal interpretation of individual-level surrogacy

ABSTRACTAt the beginning of the 21st century, a new paradigm was introduced for the evaluation of surrogate endpoints based on meta-analysis. In this paradigm, the putative surrogate is assessed at two different levels, the so-called, trial and individual level. Trial level surrogacy is defined as the association between the expected causal treatment effects across different trials populations, whereas the individual level is defined as the association between the surrogate and true endpoints, after adjusting by trial and treatment. It has been argued that the individual level surrogacy does not have a causal interpretation and, consequently, it is a poor metric of surrogacy. In the present work, an alternative definition of individual level surrogacy is introduced based on individual causal treatment effects. In addition, using the maximum entropy principle, a direct link between the individual level surrogacy, as defined in the meta-analytic approach, and the newly proposed definition is established. This new perspective sets the individual level surrogacy in a more coherent framework with respect to the trial level and bridges the two main schools of thought in this domain, namely, the causal inference and meta-analytic schools.

A Systematic Review and Recommendation for Reporting of Surrogate Endpoint Evaluation Using Meta-analyses.

BackgroundMeta-analysis of randomized controlled trials (RCTs) has been widely conducted for the evaluation of surrogate endpoints in oncology, but little attention has been given to the adequacy of reporting and interpretation. This review evaluated the reporting quality of published meta-analyses on surrogacy evaluation and developed recommendations for future reporting.MethodsWe searched PubMed through August 2017 to identify studies that evaluated surrogate endpoints using the meta-analyses of RCTs in oncology. Both individual patient data (IPD) and aggregate data (AD) meta-analyses were included for the review.ResultsEighty meta-analyses were identified: 22 used IPD and 58 used AD from multiple RCTs. We observed variability and reporting deficiencies in both IPD and AD meta-analyses, especially on reporting of trial selection, endpoint definition, study and patient characteristics for included RCTs, and important statistical methods and results. Based on these findings, we proposed a checklist and recommendations to improve completeness, consistency, and transparency of reports of meta-analytic surrogacy evaluation. We highlighted key aspects of the design and analysis of surrogate endpoints and presented explanations and rationale why these items should be clearly reported in surrogacy evaluation.ConclusionsOur reporting of surrogate endpoint evaluation using meta-analyses (ReSEEM) guidelines and recommendations will improve the quality in reporting and facilitate the interpretation and reproducibility of meta-analytic surrogacy evaluation. Also, they should help promote greater methodological consistency and could also serve as an evaluation tool in the peer review process for assessing surrogacy research.