Frequentist versus Bayesian approaches for AUC Confidence Intervals bounds

Abstract

In this paper we first present two approaches, Frequentist and Bayesian, to calculate the Confidence Interval (CI) of Area Under the Curve (AUC). The goal of this study is to compare both approaches and find out if they reveal significant differences along the sample size. We first generate a large number of hypothetical cases, based on True Negative (TN), True Positive (TP), False Positive (FP) and False Negative (FN), that lead to to specific AUC values (90, 85, 80, 75, etc.). We then use both Frequentist and Bayesian approach to calculate the AUC CI bounds, AUC <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</sub> and AUC <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</sub> , and plot them for visual comparison. Results indicate that 1) for one sample size value the Bayesian approach can have multiple AUC CI bounds values, while the Frequentist has unique set of bounds, 2) for all sample size, the AUC <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</sub> and AUC <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</sub> values using the Frequentist approach are consistently under-estimated compared to the Bayesian ones, and 3) for very large sample size both approaches converge toward same values.