HomeCirculationVol. 116, No. 6Letter by Janket et al Regarding Article, “Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction” Free AccessLetterPDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessLetterPDF/EPUBLetter by Janket et al Regarding Article, “Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction” Sok-Ja Janket Ye Shen Jukka H. Meurman Sok-Ja JanketSok-Ja Janket Department of General Dentistry, Boston University School of Dental Medicine, Boston, Mass Search for more papers by this author Ye ShenYe Shen Division of Biostatistics, Yale School of Public Health, New Haven, Conn Search for more papers by this author Jukka H. MeurmanJukka H. Meurman University of Helsinki Central Hospital, The Institute of Dentistry, Helsinki, Finland Search for more papers by this author Originally published7 Aug 2007https://doi.org/10.1161/CIRCULATIONAHA.107.714360Circulation. 2007;116:e133To the Editor:We read the article by Dr Nancy Cook with great interest.1 Although we agree with Dr Cook that the likelihood ratio test may be more precise, the C-statistic is the only statistical moiety that assesses potential clinical utility of a regression model via juxtaposition of predicted probability of true positives and false-positives.Relative risk is calculated by comparison of disease probability between those who have high levels of a biomarker and those who have low levels or none at all.2 This reference (usually the first quartile) is chosen without clinical consideration and may generate a large number of false-positives. If regression results were to be applied to clinical practice directly, this could lead to inappropriate clinical decisions (eg, vaccination or prescription of medications with adverse effects to everyone with a biomarker level above the first quartile). Therefore, prior to clinical application of the research results, it is imperative to consider not only true-positives but also accompanying false-positives. The C-statistic provides simultaneous assessment of true- and false-positives at different levels of risk factors.“No change in C-statistic,” as Cook stated, indicates that other factors already in the model have explained what this marker contributes, and/or this marker may generate high false-positives and provides no additional information. When we tested these properties with a simpler version of The Women’s Health Study,3 C-reactive protein levels above the first quartile yielded a relative risk of 2.74 and identified 86% of true-positives correctly. However, 69% of those who do not have heart disease were incorrectly categorized as heart disease patients. These results are consistent with the clinical observation that not all persons with high C-reactive protein develop cardiovascular events4 and that C-reactive protein has low specificity.5 Would clinicians prescribe statins to patients if they knew the false-positive rate is 69%?No matter how robust a biomarker appears to be in a regression analysis, it only provides the façade of a clinical picture. A sophisticated stochastical model remains theoretical, whereas the C-statistic, in spite of its limitations, gives us a glimpse of potential clinical relevance.6As a side note, the inherent properties of sensitivity/specificity of inanimate objects such as an x-ray machine do not vary. The sensitivity/specificity derived from observational studies, however, should be considered cohort-specific because data collection methods, study design, or cohort characteristics engender variability in these test properties.Sources of FundingDr Janket is supported by the National Scientist Development Grant from The American Heart Association. Dr Meurman is supported by Grant TYH 3245 from the Helsinki University Central Hospital, Finland; the Ulf Nilsonne Foundation (SalusAnsvar Prize), Stockholm, Sweden; and the Paivikki and Sakari Sohlberg Foundation.DisclosuresNone. References 1 Cook N. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115: 928–935.LinkGoogle Scholar2 Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999; 319: 1562–1565.CrossrefMedlineGoogle Scholar3 Levinson S, Elin R. What is C-reactive protein telling us about coronary artery disease? Arch Intern Med. 2002; 162: 389–392.CrossrefMedlineGoogle Scholar4 Libby P, Ridker, Maseri A. Inflammation and atherosclerosis. Circulation. 2002; 105: 1135–43.CrossrefMedlineGoogle Scholar5 Kushner I, Rzewnicki D, Samols D. What does minor elevation of C-reactive protein signify? Am J Med. 2006; 119: 166.e17–166.e28.CrossrefGoogle Scholar6 Greenland P, O’Malley PG. When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk [comment]. Arch Intern Med. 2005; 165: 2454–2456.CrossrefMedlineGoogle Scholar Previous Back to top Next FiguresReferencesRelatedDetails August 7, 2007Vol 116, Issue 6 Advertisement Article InformationMetrics https://doi.org/10.1161/CIRCULATIONAHA.107.714360PMID: 17679624 Originally publishedAugust 7, 2007 PDF download Advertisement SubjectsDiagnostic TestingEpidemiology