ROC curves for clinical prediction models part 3. The ROC plot: a picture that needs a 1000 words

Ben Van Calster,Laure Wynants,Gary S Collins,Jan Y Verbakel,Ewout W Steyerberg

doi:10.1016/j.jclinepi.2020.05.037

Abstract

In our recent perspective, we argue that receiver operating characteristic (ROC) curves add no useful information to the area under the ROC curve (AUC) for the evaluation of discriminatory ability of risk prediction models [ [1] Verbakel J.Y. Steyerberg E.W. Uno H. De Cock B. Wynants L. Collins G.S. et al. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models. J Clin Epidemiol. 2020; Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar ]. Our key argument is that a risk threshold needs to be considered if a model is used to support decision-making, while ROC curves in their standard form suppress threshold information. Despite her critical assessment of our article, Janssens raised no strong or convincing arguments against this view [ [2] Janssens A.C.J.W. The ROC plot: the picture that could be worth a 1000 words. J Clin Epidemiol. 2020; Abstract Full Text Full Text PDF Scopus (2) Google Scholar ]. Janssens states that the ROC curve is valid, and that it is subjective whether it has added value over the AUC alone: “others may benefit from seeing how and how much (little) the addition of predictors improves the discriminative ability”. However, that is exactly what the AUC quantifies, and Janssens seemingly agrees: “the AUC and ROC plot present the overall discriminative ability of prediction models”.

Highlights

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record
Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain
We argue that receiver operating characteristic (ROC) curves add no useful information to the area under the ROC curve (AUC) for the evaluation of discriminatory ability of risk prediction models.[1]

Summary

Introduction

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. Despite her critical assessment of our paper, Janssens raised no strong or convincing arguments against this view.[2] Janssens states that the ROC curve is valid, and that it is subjective whether it has added value over the AUC alone: “others may benefit from seeing how and how much (little) the addition of predictors improves the discriminative ability”. If physicians agree that only patients with a risk of 20% and higher should have lymphadenectomy, they are more worried about false negatives than about false positives.

Results

Conclusion