Principles from clinical trials relevant to clinical practice: Part II.

Abstract

HomeCirculationVol. 106, No. 9Principles From Clinical Trials Relevant to Clinical Practice: Part II Free AccessResearch ArticlePDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessResearch ArticlePDF/EPUBPrinciples From Clinical Trials Relevant to Clinical Practice: Part II Robert M. Califf and David L. DeMets Robert M. CaliffRobert M. Califf From the Duke Clinical Research Institute and the Division of Cardiology (R.M.C.), Duke University Medical Center, Durham, NC, and the Department of Biostatistics and Medical Informatics (D.L.D.), University of Wisconsin, Madison, Wis. Search for more papers by this author and David L. DeMetsDavid L. DeMets From the Duke Clinical Research Institute and the Division of Cardiology (R.M.C.), Duke University Medical Center, Durham, NC, and the Department of Biostatistics and Medical Informatics (D.L.D.), University of Wisconsin, Madison, Wis. Search for more papers by this author Originally published27 Aug 2002https://doi.org/10.1161/01.CIR.0000023218.39412.32Circulation. 2002;106:1172–1175This last installment of a 4-part series discusses the final 5 of 11 principles derived from clinical trials during the last 2 decades of cardiovascular clinical research and the need to apply them to the care of individual patients with heart diseases.Principle 7: Most Therapies Produce a Combination of Helpful and Harmful EffectsAs therapies have become more biologically potent, it has become increasingly clear that every treatment can be harmful in some patients while being beneficial in others, and often both good and bad effects occur in the same patient. With many therapies, clinical characteristics can identify patients with greater expected benefit or risk, and pharmacogenetics may offer further insight into predicting which patients will achieve the greatest benefit and which will have the greatest risk of harm. For example, we know that women are at higher risk for torsades de pointes when they are treated with QT-interval-prolonging drugs, and it is likely that a genetic predisposition may be important.1 Unfortunately, however, the relative power of these findings is limited, and we are left with broad treatment guidelines that will leave patients at risk of harm that cannot be predicted in absolute terms.The recognition that therapeutics are not commonly either “good” or “bad,” but carry a mixture of good and bad effects, has spawned the concept of risk-management in therapeutics. Each clinician has the responsibility of helping the patient place these risks and benefits in perspective when making decisions about therapeutics. Inevitably, this approach will require an improved grasp of probabilities and quantitative outcome estimates by clinicians and patients. Furthermore, there is an implied responsibility, both to participate in the generation of knowledge about risks and benefits through clinical trials and to report adverse events observed in the post-marketing period. Such adverse event reporting played the critical role in identifying cisapride as a cause of sudden death2 and recognizing that mibefradil caused intolerable toxicity when combined with a variety of other medications.3Principle 8: Most Beneficial Therapies Do Not Save Money, but They Are Incrementally Cost-EffectiveThe evolution of health systems in the United States, combined with the explosion of medical products and an aging population, has increased the focus on cost; this focus has been a critical element of therapeutics in most other countries, and the extent to which lessons are transferable from one culture to another remains a source of uncertainty because of different societal priorities, financial resources, and methods of payment. Indeed, the minimally important clinical difference can only be understood in the context of the cost paid for the benefit derived. As more expensive biological and mechanical therapies are devised, it is increasingly clear that we will not be able to offer all beneficial therapies to all patients. A well-designed prospective evaluation of cost can be a critical component of an effective assessment of therapy.The development of new therapies almost invariably adds cost, even when the therapies are highly effective, because the therapies are usually not curative: the patients eventually succumb or have progression of disease. Thus, the appropriate question usually is not, “Does this treatment save money?” but rather, “Is the extra money worth it?” In the United States, the national right to renal dialysis has set a de facto standard at an incremental cost of $70 000 per year of life saved, since this is the estimated cost of gaining an extra year of life with the use of dialysis.4On the basis of other trial evidence, the Global Utilization of Streptokinase and t-PA for Occluded Coronary Arteries I (GUSTO-I) trial hypothesized that an accelerated infusion of alteplase would reduce the risk of death by 15% on a relative scale or by 1% on an absolute scale, compared with a standard infusion of streptokinase. Considerable debate ensued as to whether a drug that cost $2000 more than the standard at the time was worth the cost for that degree of benefit. The trial found almost precisely the hypothesized benefit, yielding an increase in life expectancy of 0.15 years per patient. Although at first this degree of benefit sounded trivial, a formal cost-effectiveness study found that the cost per year of life saved was approximately $30 000, well below the amount spent to save a year of life with renal dialysis,4 a therapy routinely paid for by the government. Tracking the use of alteplase relative to streptokinase in the United States after an announcement of these findings and publication of the results demonstrated a step-up in use with the release of each portion of clinical and economic information5 (Figure 1). Even though alteplase is sold for a lower cost outside the US, streptokinase remains the most commonly used fibrinolytic agent globally, pointing out the complexity of economic analyses in terms of societal norms and the ability to pay for medical services. Download figureDownload PowerPointFigure 1. The increase in the use of alteplase relative to streptokinase with the release of each portion of clinical and economic information from the GUSTO-I trial. With the release of each major portion of data, including the economic data, the market share of alteplase increased. Reprinted with permission from Califf RM, Stump D, Topol EJ, et al. The impact of the cost-effectiveness study of GUSTO-1 on decision making with regard to fibrinolytic therapy. Am Heart J. 1999;137:S90–S93.Similarly, the addition of abciximab to the practice of percutaneous coronary intervention (PCI) has produced a clear clinical benefit, but the absolute cost is substantial.6 When this cost is transformed to the incremental cost per year of life saved, however, the incremental cost falls well within the range of acceptable therapies.4 Although coronary stents have not produced a demonstrable mortality benefit from randomized trial data, they have reduced the need for repeat revascularization, and cost-effectiveness studies have also found them to be within the realm of acceptable incremental costs in the United States.7 The one trial to look at both stenting and abciximab in a factorial design found the combination of treatments to be within the realm of incremental cost-effectiveness.8 For many effective secondary prevention therapies, the cost per year of life saved is extremely low, leading to a situation in which no financial objection can be raised at the level of the individual patient.4,9 The conduct of prospective cost studies simultaneously with definitive, pragmatic clinical outcome trials allows the clinical community to place therapeutics in perspective.On a pragmatic basis, cardiovascular practitioners must become involved at a local level to improve the broad understanding of incremental cost-effectiveness. Ideally, such efforts should focus on advocating the use of proven therapies and avoiding the use of unproven therapies when less-expensive or safer alternatives exist. How to place the well-organized data on many cardiovascular therapies against therapies for non-life-threatening problems or less well-evaluated therapies remains a dilemma. Given recent findings that will likely lead to widespread use of internal cardioverter-defibrillators, drug-coated stents, and left-ventricular assist devices, this emphasis on incremental cost-effectiveness will need considerable attention at the practitioner level.Principle 9: Applying the Results of Clinical Trials Is BeneficialThe purpose of all the effort of clinical trials is to provide evidence to allow patients, health care providers, and policy makers to make the best decisions for individual health outcomes and population benefit. As a hierarchy of evidence has been increasingly accepted,10 a process for developing clinical practice guidelines has evolved. The pyramid of clinical evidence (Figure 2) focuses on the randomized clinical trial as the most definitive guide to effective therapy, providing a basis for choosing which therapies should be recommended with specific levels of certainty. Download figureDownload PowerPointFigure 2. The pyramid of clinical evidence. In a rational, quantitative world, the recommendation for the individual patient would emanate from evidence built along the left-hand side of the pyramid. By integrating clinical trial outcome data into systematic overviews and guidelines, evidence-based clinical practice occurs. To the extent the guidelines lack empirical support with the best methodology, the pyramid is seen as less solid.In the United States, professional societies and disease-oriented organizations are charged with devising clinical practice guidelines. The government, through the Evidence-Based Practice Centers, funds the Agency for Healthcare Research and Quality to administer a system of quantitative analyses to support the development of guidelines. Thus, the American College of Cardiology, the American Heart Association, and the professional subdivisions of these organizations play a critical role in aggregating the knowledge base to devise guidelines.The American Heart Association and the American College of Cardiology have jointly devised a system for grading evidence that is similar to other national and global efforts.11,12 Two dimensions are considered: level of evidence (Grade) and type of recommendation (Class). The highest level of evidence (A) comes from multiple clinical trials with excellent design, followed by either a single clinical trial or well-designed observational studies without a clinical trial (B). When a recommendation is based on opinion or small case series, it is classified as level of evidence C. A Class I recommendation means that the approach should generally be used; Class IIa means that it is usually recommended with some uncertainty; Class IIb means the approach is generally not recommended and has some uncertainty; and a Class III recommendation means “Don’t do it!”Recently, convincing evidence has been produced showing that systematic administration of therapies demonstrated to be effective in clinical trials leads to better patient outcomes. In ST-elevation myocardial infarction, hospitals that deliver a higher proportion of appropriate aspirin, β-blockers, and angiotensin-converting enzyme inhibitors have better patient outcomes.13,14 Similar results have now been seen for patients with non-ST-elevation acute coronary syndromes.15These findings provide empirical support for a cycle of evidence to practice (Figure 3). When clinical trials demonstrate the clear benefit of a therapy, the therapy is adopted as a Class I, level of evidence A recommendation. When this happens, physicians, practice organizations, and hospitals can examine their systems to ensure that they can deliver the therapy efficiently and reliably. It can be incorporated into performance measures to quantify how often a practitioner or a system adheres to the standard. As discussed above, better adherence to standards (performance) should lead to better outcomes. Download figureDownload PowerPointFigure 3. The quantitative cycle of quality can be envisioned as driven by discovery science (both physical and behavioral), which leads to inventions that may lead to medical therapy or technology that can be evaluated in clinical trials. If the clinical trials are adequately designed and performed to produce a definitive result to inform clinical practice, a clinical practice guideline could be devised. Clear clinical practice guidelines can be used to derive performance indicators, which can be used to measure clinical performance (for example, patients with elevated LDL cholesterol should be treated with a statin). Practices with better performance as measured by adherence to performance indicators should have better outcomes, and by measuring outcomes, deviations can stimulate new discoveries and clinical trials. Adapted with permission from Garson A. President’s page: The great circle: a target for better patient care. J Am Coll Cardiol. 1999;34:294–295.However, embracing clinical practice guidelines does not detract from the importance of individual physician judgment or the variation in patient preferences.16 We lack definitive evidence in many situations, and each patient has a complex array of characteristics that cannot be captured in a clinical practice guideline. A perplexing and interesting example of the interface between clinical judgment and quantitative studies is afforded by the Bypass Angioplasty Revascularization Investigation (BARI). This trial randomized patients with multivessel coronary heart disease to either percutaneous revascularization or bypass surgery17; a prospective registry was kept of patients who were eligible for enrollment but did not enter the randomized trial. In the trial, patients with treated diabetes had better survival rates if they had surgery, leading to a health alert by the National Heart, Lung, and Blood Institute. In the registry, however, patients with treated diabetes who underwent PCI fared as well as patients who elected bypass surgery.18 This combination of randomized and registry evidence indicates that careful consideration of multiple clinical factors can identify a population of patients with multivessel disease and diabetes who do as well with PCI as with coronary artery bypass grafting, but that the majority of multivessel disease patients with diabetes do better with coronary artery bypass grafting.Principle 10: Some Areas of Cardiovascular Medicine are UnderservedThe power of medical practice based on evidence raises the issue of why we do not have adequate evidence for so many areas. In the history of pediatric cardiology, <30 randomized clinical trials have been completed. Almost nothing is known aside from anecdotal or observational experience (with all the flaws enumerated above) about the appropriate use of surgery or aggressive medical therapy in patients with valvular heart disease or endocarditis. A recent review of primary pulmonary hypertension had extensive treatment recommendations19 supported by very little information derived from randomized clinical outcome trials. Although these conditions are less common than the dominant cardiovascular problems, the experience with pediatric cancer (in which >80% of children are enrolled in clinical trials) points out that substantial evidence can be developed to support the best treatment for uncommon illnesses; tremendous gains in longevity for patients with pediatric cancers have occurred. Perhaps greater investment by the government in infrastructure to understand less common, but devastating, cardiovascular diseases would make a difference as it has in childhood cancer. Given the importance of evaluating proposed therapies, practitioners caring for patients with less common cardiovascular disease would serve their patients well by forming research networks—as in pediatric cancer—that could advance treatment. Such networks could also work with industry, the National Institutes of Health, and the Food and Drug Administration to define appropriate criteria for “orphan” drugs to be advanced into practice as they are evaluated over time to generate adequate evidence for a definitive assessment.Principle 11: Participation Is ImperativeThe overall conclusion found in the lessons from clinical trials and clinical practice is that practitioners and patients need to play as great a role as possible in generating the evidence for evidence-based medicine by participating in clinical trials. Pediatric oncology has set the pace, and major advances in the longevity of these children have resulted. Recently, patients with acute coronary syndromes have experienced major benefits that have been parlayed into measurable improvements in survival.20 These efforts are completely dependent on the volunteer, altruistic spirit of patients and the willingness of busy doctors to engage in clinical trial efforts. As the “rules of the game” are being increasingly defined in terms of conflict of interest,21 norms of payment for clinical research services and obligations under ethical constructs of human experimentation,22 a fair balance will need to be maintained to improve adherence to ethical norms while not paralyzing efforts to make progress in clinical research. This should include not only participating in individual trials, but also providing feedback on the constantly evolving process of developing better approaches to organizing trials, interpreting the results, and translating the research findings into practice.This is Part IV of a 4-part article. Parts I, II, and III appeared in the August 6, August 13, and August 20, 2002, issues of the journal, respectively (Circulation. 2002;106:746–751; Circulation. 2002;106:880–886; and Circulation. 2002;106:1015–1021).This project was supported in part by an Agency for Healthcare Research and Quality (AHRQ) Centers for Education and Research on Therapeutics (CERTs) cooperative agreement, grant No. U18HS10548. The authors thank Penny Hodgson and Karen Staman for their help with the multiple revisions of these manuscripts. They would also like to express their appreciation to the thousands of collaborators who have made these observations possible by participating in the clinical trials effort.FootnotesCorrespondence to Robert M. Califf, MD, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715. References 1 Ebert SN, Liu XK, Woosley RL. Female gender as a risk factor for drug-induced cardiac arrhythmias: evaluation of clinical and experimental evidence. J Womens Health. 1998; 7: 547–557.CrossrefMedlineGoogle Scholar2 Smalley W, Shatin D, Wysowski DK, et al. Contraindicated use of cisapride: the impact of an FDA regulatory action. JAMA. 2000; 284: 3036–3039.CrossrefMedlineGoogle Scholar3 Reimer KA, Califf RM. Good news for experimental concept but bad news for clinically effective therapy. Circulation. 1999; 99: 198–200.CrossrefMedlineGoogle Scholar4 Mark DB, Hlatky MA. Medical economics and the assessment of value in cardiovascular medicine. Part I. Circulation. 2002; 106: 516–520.LinkGoogle Scholar5 Califf RM, Stump D, Topol EJ, et al. The impact of the cost-effectiveness study of GUSTO-1 on decision making with regard to fibrinolytic therapy. Am Heart J. 1999; 137: S90–S93.CrossrefMedlineGoogle Scholar6 Young JJ, Kereiakes DJ. Abciximab: cost-effective survival advantage in clinical trials and clinical practice. Am Heart J. 2000; 140: S148–S153.CrossrefMedlineGoogle Scholar7 Cohen DJ, Breall JA, Ho KK, et al. Evaluating the potential cost-effectiveness of stenting as a treatment for symptomatic single-vessel coronary disease. Use of a decision-analytic model. Circulation. 1994; 89: 1859–1874.CrossrefMedlineGoogle Scholar8 Topol EJ, Mark DB, Lincoff AM, et al. Outcomes at 1 year and economic implications of platelet glycoprotein IIb/IIIa blockade in patients undergoing coronary stenting: results from a multicentre randomised trial. EPISTENT Investigators. Lancet. 1999; 354: 2019–2024.CrossrefMedlineGoogle Scholar9 Goldman L, Phillips KA, Coxson P, et al. The effect of risk factor reductions between 1981 and 1990 on coronary heart disease incidence, prevalence, mortality and cost. J Am Coll Cardiol. 2001; 38: 1012–1017.CrossrefMedlineGoogle Scholar10 Califf RM. Considerations in the design, conduct, and interpretation of quantitative clinical evidence. In: Topol EJ, ed. Textbook of Cardiovascular Medicine. Philadelphia, Pa: Lippinicott Raven Publishers; 1998:1063–1081.Google Scholar11 Godlee F. The Cochrane collaboration. BMJ. 1994; 309: 969–970.CrossrefMedlineGoogle Scholar12 McAlister FA, Laupacis A, Wells GA, et al. Users’ Guides to the Medical Literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA. 1999; 282: 1371–1377.CrossrefMedlineGoogle Scholar13 Allison JJ, Kiefe CI, Weissman NW, et al. Relationship of hospital teaching status with quality of care and mortality for Medicare patients with acute MI. JAMA. 2000; 284: 1256–1262.CrossrefMedlineGoogle Scholar14 Chen J, Radford MJ, Wang Y, et al. Do “America’s Best Hospitals” perform better for acute myocardial infarction? N Engl J Med. 1999; 340: 286–292.CrossrefMedlineGoogle Scholar15 Alexander KP, Shaw LJ, Shaw LK, et al. Value of exercise treadmill testing in women. J Am Coll Cardiol. 1998; 32: 1657–1664.CrossrefMedlineGoogle Scholar16 Guyatt G, Sackett D, Taylor DW, et al. Determining optimal therapy–randomized trials in individual patients. N Engl J Med. 1986; 314: 889–892.CrossrefMedlineGoogle Scholar17 The Bypass Angioplasty Revascularization Investigation (BARI) Investigators. Comparison of coronary bypass surgery with angioplasty in patients with multivessel disease. N Engl J Med. 1996; 335: 217–225.CrossrefMedlineGoogle Scholar18 BARI Investigators. Seven-year outcome in the Bypass Angioplasty Revascularization Investigation (BARI) by treatment and diabetic status. J Am Coll Cardiol. 2000; 35: 1122–1129.CrossrefMedlineGoogle Scholar19 Archer S, Rich S. Primary pulmonary hypertension: a vascular biology and translational research “work in progress.” Circulation. 2000; 102: 2781–2791.CrossrefMedlineGoogle Scholar20 Furman MI, Barnard MR, Krueger LA, et al. Circulating monocyte-platelet aggregates are an early marker of acute myocardial infarction. J Am Coll Cardiol. 2001; 38: 1002–1006.CrossrefMedlineGoogle Scholar21 Topol EJ, Califf RM, Van de Werf F, et al. Perspectives on large-scale cardiovascular clinical trials for the new millennium. Circulation. 1997; 95: 1072–1082.CrossrefMedlineGoogle Scholar22 Morse MA, Califf RM, Sugarman J. Monitoring and ensuring safety during clinical research. JAMA. 2001; 285: 1201–1205.CrossrefMedlineGoogle Scholar Previous Back to top Next FiguresReferencesRelatedDetailsCited By Califf R (2018) Future of Personalized Cardiovascular Medicine, Journal of the American College of Cardiology, 10.1016/j.jacc.2018.09.079, 72:25, (3301-3309), Online publication date: 1-Dec-2018. Califf R and Sugarman J (2015) Exploring the ethical and regulatory issues in pragmatic clinical trials, Clinical Trials, 10.1177/1740774515598334, 12:5, (436-441), Online publication date: 1-Oct-2015. Ruberu R and Fitzgerald S (2012) Clinical Practice Guidelines for Chronic Diseases—Understanding and Managing Their Contribution to Polypharmacy, Clinics in Geriatric Medicine, 10.1016/j.cger.2012.01.003, 28:2, (187-198), Online publication date: 1-May-2012. Lai W, Vetter V, Richmond M, Li J, Saul J, Mital S, Colan S, Newburger J, Sleeper L, McCrindle B, Minich L, Goldmuntz E, Marino B, Williams I, Pearson G, Evans F, Scott J and Cohen M (2011) Clinical Research Careers: Reports from a NHLBI Pediatric Heart Network Clinical Research Skills Development Conference, American Heart Journal, 10.1016/j.ahj.2010.08.032, 161:1, (13-67), Online publication date: 1-Jan-2011. Permanyer Miralda G (2010) ¿Son valores antagónicos la prevención y la compasión?, Medicina Clínica, 10.1016/j.medcli.2009.11.026, 135:14, (650-652), Online publication date: 1-Nov-2010. Tucci D, Schulz K and Witsell D (2010) Building a National Research Network for Clinical Investigations in Otology and Neurotology, Otology & Neurotology, 10.1097/MAO.0b013e3181c9940c, 31:2, (190-195), Online publication date: 1-Feb-2010. Lagrutta A and Salata J (2009) Safety Pharmacology and Regulatory Issues in the Development of Antiarrhythmic Medications Novel Therapeutic Targets for Antiarrhythmic Drugs, 10.1002/9780470561416.ch8, (233-269), Online publication date: 16-Dec-2009. Coller B and Califf R (2009) Traversing the Valley of Death: A Guide to Assessing Prospects for Translational Success, Science Translational Medicine, 10.1126/scitranslmed.3000265, 1:10, Online publication date: 9-Dec-2009. BORIANI G, DIEMBERGER I, MANTOVANI V, BIFFI M and MARTIGNANI C (2009) Remote Monitoring of Patients with an Implanted Device and Patients’ Outcomes: The Potential for “Win-Win” Dynamics, Journal of Cardiovascular Electrophysiology, 10.1111/j.1540-8167.2009.01576.x, 20:11, (1252-1254), Online publication date: 1-Nov-2009. Pfeffer M (2008) Anemia treatment in chronic kidney disease: shifting uncertainty, Heart Failure Reviews, 10.1007/s10741-008-9087-9, 13:4, (425-430), Online publication date: 1-Dec-2008. Boriani G, Biffi M, Marziali A, Diemberger I and Martignani C (2008) A changing scenario in the clinical use of implantable defibrillators: the need for long-term data on lead performance, Europace, 10.1093/europace/eun339, 11:1, (1-3) Califf R and Kramer J (2008) The balance of benefit and safety of rosiglitazone: important lessons for our system of drug development and postmarketing assessment, Pharmacoepidemiology and Drug Safety, 10.1002/pds.1617, 17:8, (782-786), Online publication date: 1-Aug-2008. MARCH J, SZATMARI P, BUKSTEIN O, CHRISMAN A, KONDO D, HAMILTON J, KREMER C and KRATOCHVIL C (2007) AACAP 2005 Research Forum: Speeding the Adoption of Evidence-Based Practice in Pediatric Psychiatry, Journal of the American Academy of Child & Adolescent Psychiatry, 10.1097/chi.0b013e318074eb48, 46:9, (1098-1110), Online publication date: 1-Sep-2007. Ghogawala Z, Coumans J, Benzel E, Stabile L and Barker F (2007) Ventral Versus Dorsal Decompression for Cervical Spondylotic Myelopathy: Surgeonsʼ Assessment of Eligibility for Randomization in a Proposed Randomized Controlled Trial, Spine, 10.1097/01.brs.0000255068.94058.8a, 32:4, (429-436), Online publication date: 1-Feb-2007. Permanyer-Miralda G and Ferreira-González I (2006) ¿Hacia la perversión de la medicina basada en la evidencia?, Medicina Clínica, 10.1157/13086852, 126:13, (497-499), Online publication date: 1-Apr-2006. Murillo H, Reece E, Snyderman R and Sung N (2006) Meeting the Challenges Facing Clinical Research: Solutions Proposed by Leaders of Medical Specialty and Clinical Research Societies, Academic Medicine, 10.1097/00001888-200602000-00002, 81:2, (107-112), Online publication date: 1-Feb-2006. March J, Chrisman A, Breland-Noble A, Clouse K, D'Alli R, Egger H, Gammon P, Gazzola M, Lin A, Mauro C, Rana A, Ravi H, Srirama M, Su H, Thrall G and van de Velde P (2005) Using and Teaching Evidence-Based Medicine: The Duke University Child and Adolescent Psychiatry Model, Child and Adolescent Psychiatric Clinics of North America, 10.1016/j.chc.2004.05.001, 14:2, (273-296), Online publication date: 1-Apr-2005. Praetorius F (2005) Evidence-based Medicine and Clinical Guidelines in Cardiology. Promoting Science, Practice, or Bureaucracy? Evidence-based Practice in Medicine and Health Care, 10.1007/3-540-27133-3_6, (45-60), . Diamond G and Kaul S (2004) Prior convictions, Journal of the American College of Cardiology, 10.1016/j.jacc.2004.01.035, 43:11, (1929-1939), Online publication date: 1-Jun-2004. Shah M and Stevenson L (2004) Searching for evidence: refractory questions in advanced heart failure, Journal of Cardiac Failure, 10.1016/j.cardfail.2003.10.006, 10:3, (210-218), Online publication date: 1-Jun-2004. March J, Silva S, Compton S, Anthony G, DeVeaugh-Geiss J, Califf R and Krishnan R (2004) The Child and Adolescent Psychiatry Trials Network (CAPTN), Journal of the American Academy of Child & Adolescent Psychiatry, 10.1097/00004583-200405000-00004, 43:5, (515-518), Online publication date: 1-May-2004. Califf R (2004) Defining The Balance Of Risk And Benefit In The Era Of Genomics And Proteomics, Health Affairs, 10.1377/hlthaff.23.1.77, 23:1, (77-87), Online publication date: 1-Jan-2004. Leventhal H and Friedman M (2004) Does Establishing Fidelity of Treatment Help in Understanding Treatment Efficacy? Comment on Bellg et al. (2004)., Health Psychology, 10.1037/0278-6133.23.5.452, 23:5, (452-456) Califf R and Muhlbaier L (2003) Health Insurance Portability and Accountability Act (HIPAA), Circulation, 108:8, (915-918), Online publication date: 26-Aug-2003. Sheps D, Freedland K, Golden R and McMahon R (2003) ENRICHD and SADHART: Implications for Future Biobehavioral Intervention Efforts, Psychosomatic Medicine, 10.1097/00006842-200301000-00001, 65:1, (1-2), Online publication date: 1-Jan-2003. Califf R (2002) Matching resources to treatment decisions for patients with acute coronary syndromes, Clinical Cardiology, 10.1002/clc.4960251303, 25:S1, (2-8), Online publication date: 1-Nov-2002. August 27, 2002Vol 106, Issue 9 Advertisement Article InformationMetrics https://doi.org/10.1161/01.CIR.0000023218.39412.32PMID: 12196347 Originally publishedAugust 27, 2002 Keywordsstatisticsoutcome assessmentcardiovascular diseasestherapytrialsPDF download Advertisement