Use Of Surrogate Outcomes Research Articles

Treatment effect bias in randomised controlled trials using surrogate outcomes: a preliminary cohort study analysis

Background Ideally, decisions on the value of health technologies should be based on evidence from well-conducted clinical trials that assess clinically important final patientrelevant outcomes, such as mortality or impaired quality of life. Pressure to reduce the delay in the availability of technologies to patients has led to an increased reliance on the use of surrogate outcomes [1]. A key tenant of surrogate outcomes is unbiased quantification of the predictive treatment effect on the final patient-relevant clinical outcomes. This study compares the treatment outcome of trials that use a surrogate versus a final patient-relevant primary outcome.

Using alternative statistical formats for presenting risks and risk reductions.

The success of evidence-based practice depends on the clear and effective communication of statistical information. To evaluate the effects of using alternative statistical presentations of the same risks and risk reductions on understanding, perception, persuasiveness and behaviour of health professionals, policy makers, and consumers. We searched Ovid MEDLINE (1966 to October 2007), EMBASE (1980 to October 2007), PsycLIT (1887 to October 2007), and the Cochrane Central Register of Controlled Trials (The Cochrane Library, 2007, Issue 3). We reviewed the reference lists of relevant articles, and contacted experts in the field. We included randomized and non-randomized controlled parallel and cross-over studies. We focused on four comparisons: a comparison of statistical presentations of a risk (eg frequencies versus probabilities) and three comparisons of statistical presentation of risk reduction: relative risk reduction (RRR) versus absolute risk reduction (ARR), RRR versus number needed to treat (NNT), and ARR versus NNT. Two authors independently selected studies for inclusion, extracted data, and assessed risk of bias. We contacted investigators to obtain missing information. We graded the quality of evidence for each outcome using the GRADE approach. We standardized the outcome effects using adjusted standardized mean difference (SMD). We included 35 studies reporting 83 comparisons. None of the studies involved policy makers. Participants (health professionals and consumers) understood natural frequencies better than probabilities (SMD 0.69 (95% confidence interval (CI) 0.45 to 0.93)). Compared with ARR, RRR had little or no difference in understanding (SMD 0.02 (95% CI -0.39 to 0.43)) but was perceived to be larger (SMD 0.41 (95% CI 0.03 to 0.79)) and more persuasive (SMD 0.66 (95% CI 0.51 to 0.81)). Compared with NNT, RRR was better understood (SMD 0.73 (95% CI 0.43 to 1.04)), was perceived to be larger (SMD 1.15 (95% CI 0.80 to 1.50)) and was more persuasive (SMD 0.65 (95% CI 0.51 to 0.80)). Compared with NNT, ARR was better understood (SMD 0.42 (95% CI 0.12 to 0.71)), was perceived to be larger (SMD 0.79 (95% CI 0.43 to 1.15)).There was little or no difference for persuasiveness (SMD 0.05 (95% CI -0.04 to 0.15)). The sensitivity analyses including only high quality comparisons showed consistent results for persuasiveness for all three comparisons. Overall there were no differences between health professionals and consumers. The overall quality of evidence was rated down to moderate because of the use of surrogate outcomes and/or heterogeneity. None of the comparisons assessed behaviourbehaviour. Natural frequencies are probably better understood than probabilities. Relative risk reduction (RRR), compared with absolute risk reduction (ARR) and number needed to treat (NNT), may be perceived to be larger and is more likely to be persuasive. However, it is uncertain whether presenting RRR is likely to help people make decisions most consistent with their own values and, in fact, it could lead to misinterpretation. More research is needed to further explore this question.

Surrogate outcomes in health technology assessment: An international comparison

Our aim was to review the recommendations given by health technology assessment (HTA) institutions in their methodological guidelines concerning the use of surrogate outcomes in their assessments. In a second step, we aimed at quantifying the role surrogate parameters take in assessment reports. We analyzed methodological papers and guidelines from HTA agencies with International Network of Agencies for Health Technology Assessment membership as well as from institutions related to pharmaceutical regulation (i.e., reimbursement, pricing). We analyzed the use of surrogate outcomes in a sample of HTA reports randomly drawn from the HTA database. We checked methods, results (including evidence tables), and conclusions sections and extracted the outcomes reported. We report descriptive statistics on the presence of surrogate outcomes in the reports. We identified thirty-four methodological guidelines, twenty of them addressing the issue of outcome parameter choice and the problematic of surrogate outcomes. Overall HTA agencies call on caution regarding the reliance on surrogate outcomes. None of the agencies has provided a list or catalog of acceptable and validated surrogate outcomes. We extracted the outcome parameter of 140 HTA reports. Only around half of the reports determined the outcomes for the assessment prospectively. Surrogate outcomes had been used in 62 percent of the reports. However, only 3.6 percent were based upon surrogate outcomes exclusively. All of them assessed diagnostic or screening technologies and the surrogate outcomes were predominantly test characteristics. HTA institutions seem to agree on a cautious approach to the use of surrogate outcomes in technology assessment. Thorough assessment of health technologies should not rely exclusively on surrogate outcomes.

Estrogens, episodic memory, and Alzheimer's disease: a critical update.

Estrogen-containing hormone therapy initiated during late postmenopause does not improve episodic memory (an important early symptom of Alzheimer's disease), and it increases dementia risk. Cognitive consequences of exogenous estrogen exposures during midlife are less certain. Observational evidence implies that use of hormone therapy at a younger age close to the time of menopause may reduce risk of Alzheimer's disease later in life. However, there are concerns that observational findings may be systematically biased. Partial insight on this critical issue may be gleaned from results of ongoing clinical trials involving midlife postmenopausal women (Early versus Late Intervention Trial with Estrogen; Kronos Early Estrogen Prevention Study). The effects of exogenous midlife estrogen exposures and Alzheimer risk can also be approached through better animal models, through carefully designed cohort studies, and through use of surrogate outcomes in randomized controlled trials in midlife women. Selective estrogen receptor modulators have the potential to affect cognitive outcomes and also merit additional study.

Use of surrogate outcomes in cost-effectiveness models: A review of United Kingdom health technology assessment reports

The aim of this study was to explore the use of surrogate outcomes--a substitute outcome that predicts final patient-related outcomes--in cost-effectiveness models (CEM) within health technology assessment (HTA) reports and provide guidance for their future use. Our sampling frame was all UK HTA Program monograph series reports published in 2005 and 2006. Reports were included if they addressed a treatment effectiveness/efficacy question and included a CEM based on a surrogate outcome. The two authors independently applied inclusion and exclusion criteria, and the following data was extracted from included reports: source of surrogate outcome, level of evidence for validation of the surrogate outcomes, methods used in report to quantify link between surrogate outcome and final outcome, and consideration of the uncertainty associated with using surrogate outcomes in the results or conclusions of report. Of 100 HTA reports, 35 complied with the inclusion criteria. Of these, four (11 percent) reports included a CEM based on a surrogate outcome. All four reports sourced treatment-related changes in surrogate outcome through a systematic review of the literature. One provided Level 1 surrogate evidence (randomized controlled trial data showing a strong association between the change in surrogate outcome and change final outcome); two reported Level 2 evidence (observational study data); and one provided Level 3 evidence (disease natural/ history data). The transparency of quantification and exploration of uncertainty of the surrogate and final outcome relationship varied considerably across all four reports. Recommendations are made for the use of surrogate outcomes in future HTA reports.

The use of surrogate outcomes in model-based cost-effectiveness analyses: a survey of UK Health Technology Assessment reports

To explore the use of surrogate outcomes in Health Technology Assessment (HTA) and provide a basis for guidance for their future use, validation and reporting. This report focuses on the role of surrogate outcomes in cost-effectiveness models (CEMs) within UK HTA Programme reports. Reports published in the UK HTA Programme monograph series in 2005 and 2006 formed the sampling frame for this study. Reports were selected on the basis that they addressed a treatment effectiveness/efficacy question, that they included a CEM and that the CEM was primarily based on a surrogate outcome. Reports addressing diagnostic, screening, aetiology, prognostic and methodological questions were excluded. Information was extracted from included reports by two reviewers using a standardised proforma. Surrogate outcomes were assessed according to two published validation frameworks [Journal of the American Medical Association (JAMA) criteria and Outcomes Measures in Rheumatology Clinical Trials (OMERACT) scoring schema]. A narrative synthesis of findings is presented in the form of tabular summaries and illustrative qualitative quotations. A total of 35 UK HTA reports published in 2005 and 2006 addressed an effectiveness/efficacy question and contained a CEM. Of these, four were found to have based their CEM on a surrogate outcome. All four reports sourced treatment-related changes in surrogate outcomes through a systematic review of the literature; however, there was some variability in the consistency and transparency by which these reports provided evidence of the validation for the surrogate-final outcome relationship. Only one of the reports undertook a systematic review to specifically seek the evidence base for the association between surrogate and final outcomes. Furthermore, this was the only report to provide level 1 surrogate-final outcome validation evidence, i.e. RCT data showing a strong association between the change in surrogate outcome (BPAR) and the change in final outcome (graft survival) at an individual patient level. This report met the JAMA criteria for acceptable evidence of a surrogate. Two reports provided level 2 evidence, i.e. observational study data showing the relationship between the surrogate and final outcome, and one report provided level 3 evidence, i.e. a review of disease natural history. None of the four reports achieved a sufficient score on the OMERACT schema to be judged to have acceptable evidence of a surrogate outcome by its authors. In this survey of UK HTA reports about 10% of the CEMs therein were explicitly based on surrogate outcomes. The strength of evidence for the surrogate-final outcome relationship, transparency of quantification and exploration of uncertainty of this relationship were found to vary considerably. Recommendations are made for the use of surrogate outcomes in future HTA reports.

Biomarkers and the Design of Clinical Trials in Cancer

Scientific innovation has promoted the rapid discovery and development of cancer-related biomarkers. This has fostered improved mechanistic understandings of disease biology, facilitated the development of novel targeted therapies and offers the potential to better define the natural history of a disease and to predict response to specific treatment. Clinical trials are evolving, from the use of exploratory 'correlative studies' that assist in the understanding of disease mechanisms, to the use of validated biomarkers that are integral to the design of definitive prospective studies. A rigorous process for biomarker development and validation, followed by evaluation in well-designed, prospective clinical trials that demonstrate improved patient outcomes is required for new biomarkers to change clinical practice. To date, the number of biomarkers considered clinically useful is disappointingly small because of conflicting conclusions generated from trials with practical and methodological limitations. This review will highlight several important aspects related to the design and analysis of clinical trials that incorporate a biomarker as a central component. First, we review biomarker development, how biomarkers may be used as targets and how biomarkers can influence methodology to optimize efficiency of drug development through the use of surrogate outcomes. Second, we focus on issues related to trials evaluating potential prognostic biomarkers and how the predictive properties of biomarkers may be used to determine which patients might optimally benefit from a specific intervention.

Surrogate outcomes in neurology, psychiatry, and psychopharmacology

A surrogate outcome can be defined as an outcome that can be observed sooner, at lower cost, or less invasively than the true outcome, and that enables valid inferences about the effect of intervention on the true outcome. There is increasing interest in the use of surrogate outcomes of treatment efficacy measurement in investigational drug trials. However, the significance of surrogate markers of treatment outcome in neurology and psychiatry has not yet been sufficiently demonstrated. Few such markers have been adequately "validated, " that is, shown to predict the effect of the treatment on the clinical outcome of interest. In this article, evidence that would support the validation of such markers is discussed. Biomarkers used during early clinical development programs of new psychotropic compounds are considered in the contexts of Parkinson's disease, affective disorder, and schizophrenia. The particular case of neuroprotective trials is exemplified by Parkinson's disease, where a biomarker substituting for a clinical measure of progression could be considered as a surrogate treatment outcome.

Use of Surrogate Outcomes for Cardiovascular Disease After Renal Transplantation

Use of Surrogate Outcomes for Cardiovascular Disease After Renal Transplantation

Home international normalized ratio monitoring: where evidence-based medicine is exemplified in the Medicare coverage process.

Nearly three years ago, the Centers for Medicare & Medicaid Services (CMS), formerly the Health Care Financing Administration, announced a new process for making national coverage determinations [1]. Prior to this time, the process had been criticized as inconsistent, closed, and based on a “blackbox” methodology. The new process is designed to be evidence-based, as well as open and inclusive, and relying upon scientific and clinical literature to make coverage determinations. One controversial issue that prompted the creation of the new process was the topic of home prothrombin international normalized ratio (INR) monitors. At that time, CMS was considering issuing a national noncoverage determination. Within the past year, a coalition of manufacturers, physicians, specialty societies, and patients successfully used the evidence-based process to obtain coverage of home prothrombin (INR) monitoring for patients with mechanical heart valves. This communication explains the reasoning behind this decision, particularly with respect to the coupling of testing and outcomes, and it also explores ways to possibly expand coverage, as additional information becomes available. A detailed explanation of this decision can be found on CMS’s website at www.hcfa.gov/coverage. The recent CMS Decision Memorandum on home INR monitoring embodies the use of a rigorous evidence-based approach to CMS policy decisionmaking. The construction of a desirable analytical framework for assessing the clinical effectiveness of any diagnostic test is predicated upon the notion that the results of such a test will influence patient outcomes. The ideal, randomized controlled trial for effectiveness occurs when results of the diagnostic test alter a treatment intervention, and the health outcomes associated with such treatment are included within the boundaries of that same study. However, there are possibilities where appropriate linkages of randomized controlled trials can demonstrate a similar, strong relationship between testing and health outcomes. In this latter instance, there is a coupling of surrogate (or intermediate outcomes) with eventual health outcomes. The presence of key clinical trials, particularly among patients with mechanical heart valves (MHVs) who require long-term anticoagulation, afforded CMS the opportunity to evaluate both types of analytical frameworks, which are described above. In the first case, the typical study design was comprised of a home INR test group vs. a “standard” care control group, and health outcomes are event rates (bleeding and thromboembolism). In the second analytical framework, a home INR test group was still compared against a standard care control group, but, in this instance, time-in-therapeutic range (TTR) was a more achievable surrogate outcome, given the much larger sample sizes needed to adequately compare event rates. However, since a separate body of literature has established a causal relationship between TTR and event rates, particularly as highlighted by Samsa and Matchar [2], CMS was able to rely upon such an indirect approach which is contingent upon the use of surrogate outcomes. It should be emphasized that CMS does not restrict its review of evidence to only include randomized controlled trials, but in this evaluation of home INR testing where several trials were available, it is reasonable to emphasize the strongest type of evidence that was considered. Studies involving diagnostic technologies are usually represented by crosssectional, two-by-two table analyses, which in spite of generating useful sensitivity and specificity values, do not often afford the opportunity to address outcomes, even those which are surrogate and usually more accessible. Although the Decision Memorandum contains further details, some representative supportive trials in the MHV setting (i.e., at least 50% MHV patients in both study arms) are summarized below. Horstkotte conducted one of the first randomized prospective trials related to home INR testing [3].

Trial Design Based on Surrogate End Points--Application to Comparison of Different Breast Screening Frequencies

This paper deals with the use of surrogates for the true end point in clinical or prevention trials to shorten the duration of the trial and to increase the power. It is shown that the estimate of the relative hazard using predicted mortality from prognostic variables has lower variance than the estimate using actual observed mortality. The strategy is applied to the design of the British Breast Screening Frequency Trial. In this situation, screening more frequently can be expected to lead to reduced mortality by increasing the number of cancers detected by screening, and by the change in characteristics of the tumours brought about by earlier detection. We demonstrate how the use of surrogate variables to predict mortality leads to a threefold reduction in the variance of the relative hazard, and a result 10 years earlier than would be provided by the use of mortality itself. We suggest that the use of surrogate outcomes, although clearly hazardous or even unacceptable for investigating issues of primary importance, may have a useful role in resolving subsidiary issues.