Dependence of the AUC of NTCP models on the observational dose-range highlights cautions in comparison of discriminative performance

Abstract

BackgroundNormal tissue complication probability (NTCP) models are probabilistic models that describe the risk of radio-induced toxicity in tissues or organs. In the field of radiotherapy, the area under the ROC curve (AUC) is widely used to estimate the performance in risk prediction of NTCP models. MethodsIn this work, we derived an analytical expression of the AUC for the logistic NTCP model in the case of both symmetrical and asymmetrical dose (to the normal tissue) windows around D50. Using numerical simulations, we studied the behavior of the AUC in general clinical settings, enforcing non-logistic NTCP models (Lyman-Kutcher-Burman and LogEUD) and including risk factors beyond the dose. We validated our findings using real-world radiotherapy data sets of prostate cancer patients. ResultsOur analytical expression of the AUC made explicit the dependence on both the steepness of the logistic curve (β) and the dose window width (w), showing that an increase of w pushes AUC towards higher values. Increasing values of the AUC with increasing values of w were consistently observed across simulated data sets with diverse clinical settings from published studies and real clinical data sets. ConclusionOur results reveal that the AUC of NTCP models inherits intrinsic characteristics from the clinical setting of the data set on which the models are developed, and warn against the use of the AUC to compare the performance of models constructed upon data from trials in which substantially different dose ranges were administered or accounting for different risk factors beyond the dose.