Evaluation Of Surrogate Endpoints Research Articles

A maximum entropy approach for the evaluation of surrogate endpoints based on causal inference

The maximum entropy principle offers a constructive criterion for setting up probability distributions on the basis of partial knowledge. In the present work, the principle is applied to tackle an important problem in the surrogate marker field, namely, the evaluation of a binary outcome as a putative surrogate for a binary true endpoint within a causal inference framework. In the first step, the maximum entropy principle is used to determine the relative frequencies associated with the values of the vector of potential outcomes. Subsequently, in the second step, these relative frequencies are used in combination with two newly proposed metrics of surrogacy, the so-called individual causal association and the surrogate predictive function, to assess the validity of the surrogate. The procedure is conceptually similar to the use of noninformative or reference priors in Bayesian statistics. Additionally, approximate, identifiable bounds are proposed for the estimands of interest, and their performance is studied via simulations. The methods are illustrated using data from a clinical trial involving schizophrenic patients, and a newly developed and user-friendly R package Surrogate is provided to carry out the validation exercise.

Evaluation of six months sputum culture conversion as a surrogate endpoint in a multidrug resistant-tuberculosis trial

The emergence of multidrug resistant-tuberculosis (MDR-TB), defined as Mycobacterium tuberculosis strains with in vitro resistance to at least isoniazid and rifampicin, has necessitated evaluation and validation of appropriate surrogate endpoints for treatment response in drug trials for MDR-TB. The trial that has demonstrated efficacy of bedaquiline, a diarylquinoline that inhibits mycobacterial ATP synthase, possesses the requisite features to conduct this evaluation. Approval of bedaquiline for use in MDR-TB was based primarily on the results of the controlled C208 Stage II study (ClinicalTrials.gov number, NCT00449644) including 160 patients randomized 1:1 to receive bedaquiline or placebo for 24 weeks when added to an 18–24-month preferred five-drug background regimen. Since randomization in C208 Stage II was preserved until study end, the trial results allow for the investigation of the complex relationship between sustained durable outcome with either Week 8 or Week 24 culture conversion as putative surrogate endpoints. The relationship between Week 120 outcome with Week 8 or Week 24 culture conversion was investigated using a descriptive analysis and with a recently developed statistical methodology for surrogate endpoint evaluation using methods of causal inference.The results demonstrate that sputum culture conversion at 24 weeks is more reliable than sputum culture conversion at 8 weeks when assessing the outcome of adding one new drug to a MDR-TB regimen.

Validation of surrogate endpoints in cancer clinical trials via principal stratification with an application to a prostate cancer trial.

Increasing attention has been focused on the use and validation of surrogate endpoints in cancer clinical trials. Previous literature on validation of surrogate endpoints are classified into four approaches: the proportion explained approach; the indirect effects approach; the meta-analytic approach; and the principal stratification approach. The mainstream in cancer research has seen the application of a meta-analytic approach. However, VanderWeele (2013) showed that all four of these approaches potentially suffer from the surrogate paradox. It was also shown that, if a principal surrogate satisfies additional criteria called one-sided average causal sufficiency, the surrogate cannot exhibit a surrogate paradox. Here, we propose a method for estimating principal effects under a monotonicity assumption. Specifically, we consider cancer clinical trials which compare a binary surrogate endpoint and a time-to-event clinical endpoint under two naturally ordered treatments (e.g. combined therapy vs. monotherapy). Estimation based on a mean score estimating equation will be implemented by the expectation-maximization algorithm. We will also apply the proposed method as well as other surrogacy criteria to evaluate the surrogacy of prostate-specific antigen using data from a phase III advanced prostate cancer trial, clarifying the complementary roles of both the principal stratification and meta-analytic approaches in the evaluation of surrogate endpoints in cancer. Copyright © 2017 John Wiley & Sons, Ltd.

Assessing a surrogate predictive value: a causal inference approach.

Several methods have been developed for the evaluation of surrogate endpoints within the causal-inference and meta-analytic paradigms. In both paradigms, much effort has been made to assess the capacity of the surrogate to predict the causal treatment effect on the true endpoint. In the present work, the so-called surrogate predictive function (SPF) is introduced for that purpose, using potential outcomes. The relationship between the SPF and the individual causal association, a new metric of surrogacy recently proposed in the literature, is studied in detail. It is shown that the SPF, in conjunction with the individual causal association, can offer an appealing quantification of the surrogate predictive value. However, neither the distribution of the potential outcomes nor the SPF are identifiable from the data. These identifiability issues are tackled using a two-step procedure. In the first step, the region of the parametric space of the distribution of the potential outcomes, compatible with the data at hand, is geometrically characterized. Further, in a second step, a Monte Carlo approach is used to study the behavior of the SPF on the previous region. The method is illustrated using data from a clinical trial involving schizophrenic patients and a newly developed and user friendly R package Surrogate is provided to carry out the validation exercise. Copyright © 2016 John Wiley & Sons, Ltd.

An information-theoretic approach for the evaluation of surrogate endpoints based on causal inference.

In this work a new metric of surrogacy, the so-called individual causal association (ICA), is introduced using information-theoretic concepts and a causal inference model for a binary surrogate and true endpoint. The ICA has a simple and appealing interpretation in terms of uncertainty reduction and, in some scenarios, it seems to provide a more coherent assessment of the validity of a surrogate than existing measures. The identifiability issues are tackled using a two-step procedure. In the first step, the region of the parametric space of the distribution of the potential outcomes, compatible with the data at hand, is geometrically characterized. Further, in a second step, a Monte Carlo approach is proposed to study the behavior of the ICA on the previous region. The method is illustrated using data from the Collaborative Initial Glaucoma Treatment Study. A newly developed and user-friendly R package Surrogate is provided to carry out the evaluation exercise.

Immunological surrogate endpoints to evaluate vaccine efficacy

An immunological surrogate endpoints is a vaccine-induced immune response (either humoral or cellular immune) that predicts protection against clinical endpoints (infection or disease), and can be used to evaluate vaccine efficacy in clinical vaccine trials. Compared with field efficacy trials observing clinical endpoints, immunological vaccine trials could reduce the sample size or shorten the duration of a trial, which promote the license and development of new candidate vaccines. For these reasons, establishing immunological surrogate endpoints is one of 14 Grand Challenges of Global Health of the National Institutes of Health (NIH) and the Bill and Melinda Gates Foundation. From two parts of definition and statistical methods for evaluation of surrogate endpoints, this review provides a more comprehensive description.

Exploring the relationship between the causal-inference and meta-analytic paradigms for the evaluation of surrogate endpoints.

Nowadays, two main frameworks for the evaluation of surrogate endpoints, based on causal-inference and meta-analysis, dominate the scene. Earlier work showed that the metrics of surrogacy introduced in both paradigms are related, although in a complex way that is difficult to study analytically. In the present work, this relationship is further examined using simulations and the analysis of a case study. The results indicate that the extent to which both paradigms lead to similar conclusions regarding the validity of the surrogate, depends on a complex interplay between multiple factors like the ratio of the between and within trial variability and the unidentifiable correlations between the potential outcomes. All the analyses were carried out using the newly developed R package Surrogate, which is freely available via CRAN.

Principal Causal Effect Identification and Surrogate end point Evaluation by Multiple Trials

Summary Principal stratification is a causal framework to analyse randomized experiments with a post-treatment variable between the treatment and end point variables. Because the principal strata defined by the potential outcomes of the post-treatment variable are not observable, we generally cannot identify the causal effects within principal strata. Motivated by a real data set of phase III adjuvant colon cancer clinical trials, we propose approaches to identifying and estimating the principal causal effects via multiple trials. For the identifiability, we remove the commonly used exclusion restriction assumption by stipulating that the principal causal effects are homogeneous across these trials. To remove another commonly used monotonicity assumption, we give a necessary condition for the local identifiability, which requires at least three trials. Applying our approaches to the data from adjuvant colon cancer clinical trials, we find that the commonly used monotonicity assumption is untenable, and disease-free survival with 3-year follow-up is a valid surrogate end point for overall survival with 5-year follow-up, which satisfies both causal necessity and causal sufficiency. We also propose a sensitivity analysis approach based on Bayesian hierarchical models to investigate the effect of the deviation from the homogeneity assumption.

Statistical evaluation of surrogate endpoints with examples from cancer clinical trials.

A surrogate endpoint is intended to replace a clinical endpoint for the evaluation of new treatments when it can be measured more cheaply, more conveniently, more frequently, or earlier than that clinical endpoint. A surrogate endpoint is expected to predict clinical benefit, harm, or lack of these. Besides the biological plausibility of a surrogate, a quantitative assessment of the strength of evidence for surrogacy requires the demonstration of the prognostic value of the surrogate for the clinical outcome, and evidence that treatment effects on the surrogate reliably predict treatment effects on the clinical outcome. We focus on these two conditions, and outline the statistical approaches that have been proposed to assess the extent to which these conditions are fulfilled. When data are available from a single trial, one can assess the "individual level association" between the surrogate and the true endpoint. When data are available from several trials, one can additionally assess the "trial level association" between the treatment effect on the surrogate and the treatment effect on the true endpoint. In the latter case, the "surrogate threshold effect" can be estimated as the minimum effect on the surrogate endpoint that predicts a statistically significant effect on the clinical endpoint. All these concepts are discussed in the context of randomized clinical trials in oncology, and illustrated with two meta-analyses in gastric cancer.

Surrogate Endpoint Evaluation: Principal Stratification Criteria and the Prentice Definition.

A common problem of interest within a randomized clinical trial is the evaluation of an inexpensive response endpoint as a valid surrogate endpoint for a clinical endpoint, where a chief purpose of a valid surrogate is to provide a way to make correct inferences on clinical treatment effects in future studies without needing to collect the clinical endpoint data. Within the principal stratification framework for addressing this problem based on data from a single randomized clinical efficacy trial, a variety of definitions and criteria for a good surrogate endpoint have been proposed, all based on or closely related to the "principal effects" or "causal effect predictiveness (CEP)" surface. We discuss CEP-based criteria for a useful surrogate endpoint, including (1) the meaning and relative importance of proposed criteria including average causal necessity (ACN), average causal sufficiency (ACS), and large clinical effect modification; (2) the relationship between these criteria and the Prentice definition of a valid surrogate endpoint; and (3) the relationship between these criteria and the consistency criterion (i.e., assurance against the "surrogate paradox"). This includes the result that ACN plus a strong version of ACS generally do not imply the Prentice definition nor the consistency criterion, but they do have these implications in special cases. Moreover, the converse does not hold except in a special case with a binary candidate surrogate. The results highlight that assumptions about the treatment effect on the clinical endpoint before the candidate surrogate is measured are influential for the ability to draw conclusions about the Prentice definition or consistency. In addition, we emphasize that in some scenarios that occur commonly in practice, the principal strata sub-populations for inference are identifiable from the observable data, in which cases the principal stratification framework has relatively high utility for the purpose of effect modification analysis, and is closely connected to the treatment marker selection problem. The results are illustrated with application to a vaccine efficacy trial, where ACN and ACS for an antibody marker are found to be consistent with the data and hence support the Prentice definition and consistency.

Evaluating surrogate endpoints, prognostic markers, and predictive markers: Some simple themes.

A surrogate endpoint is an endpoint observed earlier than the true endpoint (a health outcome) that is used to draw conclusions about the effect of treatment on the unobserved true endpoint. A prognostic marker is a marker for predicting the risk of an event given a control treatment; it informs treatment decisions when there is information on anticipated benefits and harms of a new treatment applied to persons at high risk. A predictive marker is a marker for predicting the effect of treatment on outcome in a subgroup of patients or study participants; it provides more rigorous information for treatment selection than a prognostic marker when it is based on estimated treatment effects in a randomized trial. We organized our discussion around a different theme for each topic. "Fundamentally an extrapolation" refers to the non-statistical considerations and assumptions needed when using surrogate endpoints to evaluate a new treatment. "Decision analysis to the rescue" refers to use the use of decision analysis to evaluate an additional prognostic marker because it is not possible to choose between purely statistical measures of marker performance. "The appeal of simplicity" refers to a straightforward and efficient use of a single randomized trial to evaluate overall treatment effect and treatment effect within subgroups using predictive markers. The simple themes provide a general guideline for evaluation of surrogate endpoints, prognostic markers, and predictive markers.

Hard, harder, hardest: principal stratification, statistical identifiability, and the inherent difficulty of finding surrogate endpoints.

In many areas of clinical investigation there is great interest in identifying and validating surrogate endpoints, biomarkers that can be measured a relatively short time after a treatment has been administered and that can reliably predict the effect of treatment on the clinical outcome of interest. However, despite dramatic advances in the ability to measure biomarkers, the recent history of clinical research is littered with failed surrogates. In this paper, we present a statistical perspective on why identifying surrogate endpoints is so difficult. We view the problem from the framework of causal inference, with a particular focus on the technique of principal stratification (PS), an approach which is appealing because the resulting estimands are not biased by unmeasured confounding. In many settings, PS estimands are not statistically identifiable and their degree of non-identifiability can be thought of as representing the statistical difficulty of assessing the surrogate value of a biomarker. In this work, we examine the identifiability issue and present key simplifying assumptions and enhanced study designs that enable the partial or full identification of PS estimands. We also present example situations where these assumptions and designs may or may not be feasible, providing insight into the problem characteristics which make the statistical evaluation of surrogate endpoints so challenging.

Impact of Copula Directional Specification on Multi-Trial Evaluation of Surrogate End Points

Evaluation of surrogate end points using patient-level data from multiple trials is the gold standard, where multi-trial copula models are used to quantify both patient-level and trial-level surrogacy. While limited consideration has been given in the literature to copula choice (e.g., Clayton), no prior consideration has been given to direction of implementation (via survival vs. distribution functions). We demonstrate that even with the “correct” copula family, directional misspecification leads to biased estimates of patient-level and trial-level surrogacy. We illustrate with a simulation study and a reanalysis of disease-free survival as a surrogate for overall survival in early stage colon cancer.

Center-within-trial versus trial-level evaluation of surrogate endpoints

Evaluation of candidate surrogate endpoints using individual patient data from multiple clinical trials is considered the gold standard approach to validate surrogates at both patient and trial levels. However, this approach assumes the availability of patient-level data from a relatively large collection of similar trials, which may not be possible to achieve for a given disease application. One common solution to the problem of too few similar trials involves performing trial-level surrogacy analyses on trial sub-units (e.g., centers within trials), thereby artificially increasing the trial-level sample size for feasibility of the multi-trial analysis. To date, the practical impact of treating trial sub-units (centers) identically to trials in multi-trial surrogacy analyses remains unexplored, and conditions under which this ad hoc solution may in fact be reasonable have not been identified. We perform a simulation study to identify such conditions, and demonstrate practical implications using a multi-trial dataset of patients with early stage colon cancer.

Surrogate end points: when should they be used?

Surrogate end point evaluation has received a lot of attention for approximately a quarter of a century. Throughout this time, the topic has been surrounded with both hopes and perils. The history of the evaluation of surrogate end point evaluation is described, from Prentice’s definition, via single-trial evaluation to meta-analytic approaches. While based on technical developments and advances in statistical methodology, the treatment of the topic here is largely nontechnical. A perspective is given as to what might be a sensible way forward. It is clear that statistical evaluation can offer a contribution to the discussion as to whether a surrogate will be adopted and in what form, but can never be seen as the sole decision maker. While there is room for the adoption of surrogates, it is very important that pitfalls and drawbacks be kept in mind at all times.

Statistical evaluation of surrogate endpoints: a systematic review

Results Prentice’s operational criteria, proportion of treatment effect explained and relative effect were the original techniques for evaluating surrogacy. These led to the development of several approaches for surrogate evaluation, namely; principal stratification, direct and indirect effects, meta-analytical (including the surrogate threshold effect) and information theory. Each methodological approach raises various computational issues in practice and conceptual issues regarding causality, transportability and interpretation.

A unified procedure for meta-analytic evaluation of surrogate end points in randomized clinical trials

The meta-analytic approach to evaluating surrogate end points assesses the predictiveness of treatment effect on the surrogate toward treatment effect on the clinical end point based on multiple clinical trials. Definition and estimation of the correlation of treatment effects were developed in linear mixed models and later extended to binary or failure time outcomes on a case-by-case basis. In a general regression setting that covers nonnormal outcomes, we discuss in this paper several metrics that are useful in the meta-analytic evaluation of surrogacy. We propose a unified 3-step procedure to assess these metrics in settings with binary end points, time-to-event outcomes, or repeated measures. First, the joint distribution of estimated treatment effects is ascertained by an estimating equation approach; second, the restricted maximum likelihood method is used to estimate the means and the variance components of the random treatment effects; finally, confidence intervals are constructed by a parametric bootstrap procedure. The proposed method is evaluated by simulations and applications to 2 clinical trials.

Bayesian Adaptive Trial Design for a Newly Validated Surrogate Endpoint

The evaluation of surrogate endpoints for primary use in future clinical trials is an increasingly important research area, due to demands for more efficient trials coupled with recent regulatory acceptance of some surrogates as 'valid.' However, little consideration has been given to how a trial that utilizes a newly validated surrogate endpoint as its primary endpoint might be appropriately designed. We propose a novel Bayesian adaptive trial design that allows the new surrogate endpoint to play a dominant role in assessing the effect of an intervention, while remaining realistically cautious about its use. By incorporating multitrial historical information on the validated relationship between the surrogate and clinical endpoints, then subsequently evaluating accumulating data against this relationship as the new trial progresses, we adaptively guard against an erroneous assessment of treatment based upon a truly invalid surrogate. When the joint outcomes in the new trial seem plausible given similar historical trials, we proceed with the surrogate endpoint as the primary endpoint, and do so adaptively-perhaps stopping the trial for early success or inferiority of the experimental treatment, or for futility. Otherwise, we discard the surrogate and switch adaptive determinations to the original primary endpoint. We use simulation to test the operating characteristics of this new design compared to a standard O'Brien-Fleming approach, as well as the ability of our design to discriminate trustworthy from untrustworthy surrogates in hypothetical future trials. Furthermore, we investigate possible benefits using patient-level data from 18 adjuvant therapy trials in colon cancer, where disease-free survival is considered a newly validated surrogate endpoint for overall survival.

Simplified modeling strategies for surrogate validation with multivariate failure-time data

The linear mixed effects model has become a standard tool for the analysis of continuous hierarchical data such as, for example, repeated measures or data from meta-analyses. However, in certain situations the model does pose unavoidable computational problems. In the context of surrogate markers, this problem has appeared when using an estimation and prediction-based approach for the evaluation of surrogate endpoints. Convergence problems can occur mainly due to small between-trial variability or small number of trials. A number of alternative strategies has been proposed and studied for normally distributed data, but not such study has been conducted for other types of endpoints. The idea is to study if such simplified strategies, which always ignore individual level surrogacy, can also be applied when both surrogate and true endpoints are of failure-time types. It is shown via simulations that the 3 simplified strategies produced biased estimates, especially for the cases in which the strength of individual level association is different from the strength of trial level association. For this reason, it is recommended not to use simplified strategies when dealing with failure-time data, in contrast to the case of normally distributed data, for which simplified strategies are recommended. Possible reasons for this discrepancy might be that, in this case, ignoring the individual level association influences estimates of the mean structure parameters, what results in distorted estimates of the trial level association.

A unified framework for the evaluation of surrogate endpoints in mental-health clinical trials

For a number of reasons, surrogate endpoints are considered instead of the so-called true endpoint in clinical studies, especially when such endpoints can be measured earlier, and/or with less burden for patient and experimenter. Surrogate endpoints may occur more frequently than their standard counterparts. For these reasons, it is not surprising that the use of surrogate endpoints in clinical practice is increasing. Building on the seminal work of Prentice(1) and Freedman et al.,(2) Buyse et al. (3) framed the evaluation exercise within a meta-analytic setting, in an effort to overcome difficulties that necessarily surround evaluation efforts based on a single trial. In this article, we review the meta-analytic approach for continuous outcomes, discuss extensions to non-normal and longitudinal settings, as well as proposals to unify the somewhat disparate collection of validation measures currently on the market. Implications for design and for predicting the effect of treatment in a new trial, based on the surrogate, are discussed. A case study in schizophrenia is analysed.