How Accurate Are 3 Risk Prediction Models in US Women?

Abstract

HomeCirculationVol. 125, No. 14How Accurate Are 3 Risk Prediction Models in US Women? Free AccessEditorialPDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessEditorialPDF/EPUBHow Accurate Are 3 Risk Prediction Models in US Women? Erin D. Michos, MD, MHS and Roger S. Blumenthal, MD Erin D. MichosErin D. Michos From The Johns Hopkins Ciccarone Center for the Prevention of Heart Disease, Baltimore, MD. Search for more papers by this author and Roger S. BlumenthalRoger S. Blumenthal From The Johns Hopkins Ciccarone Center for the Prevention of Heart Disease, Baltimore, MD. Search for more papers by this author Originally published7 Mar 2012https://doi.org/10.1161/CIRCULATIONAHA.112.099929Circulation. 2012;125:1723–1726Other version(s) of this articleYou are viewing the most recent version of this article. Previous versions: January 1, 2012: Previous Version 1 Cardiovascular disease (CVD) is the leading cause of death for US women, and nearly two thirds who died suddenly of CVD had no previous symptoms.1 Therefore, it is of great importance to identify “at-risk” women early, so that effective primary prevention strategies can be instituted. A universal recommendation of prevention guidelines is that all asymptomatic women should undergo a global risk assessment.2Article see p 1748The Adult Treatment Panel III (ATP-III) of the National Cholesterol Education Program set thresholds for lipid treatment based on one's predicted 10-year risk for a hard coronary heart disease (CHD) event (myocardial infarction or CHD death) using a modified Framingham risk score (FRS).3 There are limitations, however, with use of the ATP-III model for CVD prediction in women. The ATP-III version predicts “hard” CHD events but not angina or revascularizations, even if such revascularizations are performed to manage an acute coronary syndrome. Yet women are more likely to experience “soft” CHD events or strokes than men. Moreover, a single laboratory measurement does not reflect lipid values over a woman's lifetime. The FRS does not take into account family history of premature CHD, a well-established independent risk factor for CHD events.4Also, because stroke accounts for a higher proportion of CVD events than CHD in women before age 75 years,1 we believe that total CVD risk should be the outcome for prediction and preventive strategies. Indeed, that is the conclusion of the 2011 American Heart Association CVD prevention guidelines for women.5 Furthermore, the FRS only predicts 10-year risk, whereas for women, the issue is most often “lifetime” risk. The Third National Health and Nutrition Examination Survey found that among women without a history of CHD or diabetes, 92% of those aged 60 to 69 years and >98% aged <59 years had a low-risk FRS <10%,6 yet a woman's lifetime risk of CVD is significantly higher (an estimated 40%).7To address these potential limitations, other, newer risk models have emerged that include FRS variations developed for total CVD and for 30-year risk, as well as a nonlaboratory model (which substitutes body mass index for lipid variables),8 and the Reynolds risk score (RRS) for CVD9 (Table).Table. Comparison of Risk Prediction ModelsTimePredictorsOutcome†ATP-III FRS*3 (diabetes=CHD risk equivalent)10 yAge, total cholesterol, HDL-C, SBP, antihypertensive therapy, smokingMI, CHD deathFRS total CVD*810 yAge, diabetes, smoking, treated and untreated SBP, total cholesterol, HDL-CMI, CHD death, coronary insufficiency, angina, ischemic stroke, hemorrhagic stroke, transient ischemic attack, peripheral artery disease, heart failureFRS total CVD, nonlaboratory*810 yAs for FRS, total CVD, with BMI replacing lipidsSame as FRS: total CVDFRS long-term CVD830 yMale sex, age, SBP, antihypertensive therapy, smoking, diabetes, total cholesterol, HDL-CSame as FRS: total CVDRRS*910 yAge, SBP, hs-CRP, total cholesterol, HDL-C, hemoglobin A1c (if diabetic), smoking, FH of premature MIMI, stroke, coronary revascularization, CVD deathATP-III indicates Adult Treatment Panel III; FRS, Framingham Risk Score; CHD, coronary heart disease; HDL-C, high-density lipoprotein cholesterol; SBP, systolic blood pressure; MI, myocardial infarction; CVD, cardiovascular disease; BMI, body mass index; RRS, Reynolds Risk Score; hs-CRP, high-sensitivity C-reactive protein; and FH, family history.*Sex-specific models.†Outcome used in Women's Health Initiative Observational Study: CHD, ischemic stroke, and CVD death.The RRS only incorporates 3 new variables into the ATP-III version (family history of premature CHD, high-sensitivity C-reactive protein [hs-CRP], and hemoglobin A1c for those with diabetes). In the Women's Health Study, the RRS reclassified ≈40% of women at intermediate risk by ATP-III into higher or lower categories with improved accuracy.9 The 2010 American Heart Association/American College of Cardiology guidelines subsequently gave hs-CRP screening a class IIa indication for asymptomatic women ≥60 years old and a class IIb recommendation for younger women at intermediate risk.2All of these risk models were derived with slightly different cardiovascular end points (Table), which makes them difficult to compare, and they were developed in almost exclusively white populations. Clinicians need to decide which patients are potential candidates for more aggressive primary preventive therapy (such as use of statins and aspirin). A critical question is which risk prediction model clinicians should use routinely in clinical practice.In this issue of Circulation, Cook and colleagues10 compared predicted risk using 3 existing models (FRS ATP-III for hard CHD, FRS for total CVD, and RRS for CVD) against observed event rates in another validation cohort, the Women's Health Initiative Observational Study, a large, racially/ethnically diverse cohort of US women. Because laboratory assays were newly measured, a case-cohort design was used for efficiency, with the random subsample directly representative of the full cohort.Understanding the statistical modeling used in this report is complex, but several key points emerge: (1) No new models were derived; only existing models were validated. (2) Because models predict differing outcomes, models were recalibrated to have the same CVD end point, which changes the intercept of the model but not the risk factor coefficients.As demonstrated in Figure 2 of the report by Cook et al,10 which plots average predicted risk within decile against observed risk in that decile, RRS had the best agreement. This improved discrimination was consistent for both white and black women. The ATP-III model had good prediction for total CVD, although it was designed to predict hard CHD only. The ATP-III model overestimated actual CHD risk, and the Framingham CVD score overestimated total CVD.There was a nominal improvement in the C statistic (0.765 versus 0.757 for RRS compared with ATP-III), which is statistically significant but disappointingly small. Using the RRS instead of ATP-III, Cook et al10 indicate a substantial number of women would be reclassified. For the 5% of women at ATP-III risk of 10% to 20% (intermediate/moderately high risk), 23% would be reclassified as ≥20% risk and 18% as <10% risk. Of those originally predicted to have 5% to 10% 10-year CHD risk (moderate risk) on the basis of ATP-III, 5% would be reclassified as at high risk (>20%) and 29% at >10%. Because 8 to 10 million American women have an ATP-III 10-year risk of 5% to 20%, application of the RRS could have a major impact on CVD prevention in women.9This finding has important clinical implications that would change low-density lipoprotein cholesterol targets for initiation of pharmacotherapy and would alter low-density lipoprotein cholesterol treatment goals for a substantial number of US women. One hopes that this improved classification of risk and resulting modification of treatment strategies will lead to improved cardiovascular outcomes; however, we do not have outcome data yet for this approach.The net reclassification index (NRI) is a relatively new statistical measure that describes the number or percentage of participants who were reclassified appropriately (ie, to a lower-risk group for nonevents or to a higher-risk group for events) and inappropriately (to a lower-risk group for events or to a higher-risk group for nonevents). In this study, compared with the ATP-III score, the RRS had an NRI of 4.9%, which is modest. Consider that coronary artery calcium (CAC), a measure of subclinical atherosclerosis burden, has a much larger NRI of 25% using population-based data from the Multi-Ethnic Study of Atherosclerosis (MESA) and 55% for the intermediate-risk group.11In MESA, hs-CRP levels ≥2 mg/L did not independently predict events.12 Among MESA participants who would have met the age, low-density lipoprotein cholesterol, and hs-CRP eligibility criteria of the JUPITER trial (Justification for the Use of statins in Prevention: an Intervention Trial Evaluating Rosuvastatin), nearly half had CAC of 0 with a very low absolute event rate and a high estimated number needed to treat, with the majority of events occurring among the 25% of participants with CAC scores >100, which suggests that CAC identifies subgroups expected to derive the most and least absolute benefit from statin treatment.12 Of note, in MESA, the RRS (which includes family history) predicted progression of CAC and clinical events better than ATP-III FRS when there was discordance between the scoring systems.13Although the present validation study by Cook et al10 suggests the FRS-based models overestimate predicted risk, some studies conversely have suggested the ATP-III score may underestimate risk in some women. In a study of women <65 years of age who presented with their first myocardial infarction, none had a prior ATP-III–predicted risk of >20%; only 5% were at intermediate risk, with 95% being calculated as being at low risk and only 18% meeting National Cholesterol Education Program criteria for lipid lowering.14 We previously evaluated ATP-III–predicted risk across a continuum of CAC scores in 3 studies of middle-aged asymptomatic nondiabetic women and found the majority (>90%) were classified as low risk (<10%) by ATP-III, yet a substantial number had a significant burden of subclinical atherosclerosis.15–17 These studies suggest missed opportunities to initiate aggressive preventive lifestyle and appropriate pharmacological strategies.Like hs-CRP, assessment of ankle-brachial index, carotid intima-medial thickness, and CAC were also given class IIa indications in the American Heart Association/American College of Cardiology Foundation guidelines for screening consideration in asymptomatic intermediate-risk individuals to help refine risk prediction.2 The RRS does not include assessment of subclinical atherosclerosis burden or exercise capacity, which both predict CVD risk in women above traditional risk factor assessment.18–20 We currently do not have a 10-year risk prediction model available that incorporates these imaging measures, although MESA does have a CAC-based “arterial age” calculator.21 The ATP-III report did indicate that an intermediate-risk patient with a CAC score >75th percentile could be elevated into a high-risk category.3The present report by Cook et al10 is a highly statistical paper that compares 3 commonly used risk prediction models, but one might ask whether these risk prediction models are overly complex for clinical use. The ATP-III model of performing a global risk assessment and targeting varying low-density lipoprotein cholesterol thresholds based on one's predicted risk is tedious, and many clinicians do not take the time to calculate FRS or RRS in their office despite available Web-based tools. A simpler algorithm of whom to select for lipid-lowering therapy may be warranted. Furthermore, the major randomized clinical trials of statins for primary prevention did not use a set FRS or RRS threshold for study eligibility, which makes low-density lipoprotein target guidelines based on global risk model thresholds slightly discordant from clinical trial enrollment, although the basic tenets are the same (as estimated risk increases, the absolute benefit will be greater).Additionally, the study by Cook et al10 does not address how to best incorporate lifetime risk into our considerations for risk prediction, particularly in regard to eligibility for pharmacotherapy. The 10-year RRS still focuses on short-term risk, and younger individuals typically have very low 10-year risks but still have substantial lifetime risks. “Low risk” by these 10-year calculators is not the same as “no risk,” but the label of low-risk status can potentially lead to false reassurance and lower one's motivation to engage in lifestyle modifications. In an analysis of adherence to preventive guidelines, patients with a low FRS were significantly less likely to receive counseling regarding physical activity and diet than patients at intermediate or high FRS.22Clearly, we all agree on the importance of preventive efforts to achieve lifestyle modifications as paramount for all women. Potent statin therapy is warranted in patients at high short-term risk. In other patients, we submit that lifetime risk should factor into decision making. If lifetime risk is high, this concept might motivate patients toward lifestyle changes, and some patients with high lifetime risk may also benefit from statin therapy. Patients at low short-term and low lifetime risk typically can focus on maintenance of ideal cardiovascular health.In conclusion, studies examining the ATP-III risk prediction model in women have highlighted problems with both underestimation and overestimation of risk. Clearly, the time has come for improved risk stratification to better identify those who will most and least benefit from intensified primary prevention. The report by Cook et al10 suggests that among ethnically diverse US women, the RRS is a more accurate predictor of short-term 10-year risk, but the NRI is still only modest. Many questions remain, however, including how best and when to incorporate other measures (such as subclinical vascular disease and exercise testing) into risk prediction, how best to incorporate lifetime risk assessments, and whether making treatment decisions (ie, initiation or titration of statins) based on RRS-predicted level of risk actually changes outcomes compared with current models. We congratulate the authors for tackling this complicated issue. Nevertheless, having too much confidence in cardiovascular risk prediction does indeed appear to be a risky proposition.DisclosuresNone.FootnotesThe opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.Correspondence to Erin D. Michos, MD, MHS, Johns Hopkins Ciccarone Preventive Cardiology Center, Carnegie 568, 600 N Wolfe St, Baltimore, MD 21287. E-mail [email protected]edu