Using ANOVA/random-effects variance estimates to compute a two-sampleU-statistic of order (1,1) estimate of variance

Abstract

ABSTRACTThe classical empirical, area under the receiver operating characteristic (ROC) curve (AUC) is a two-sample U-statistic of order (1,1). Its variance can be written out as a sum of three tractable covariances. It is then possible to consider each of these covariances as U-statistics themselves and follow the U-statistics formalism to derive their unbiased estimates. Over the years, alternative methods have been proposed to estimate the variance of AUC. For example, Delong et al. have proposed a straightforward estimate that has attractive asymptotic properties. At small sample sizes, however, the DeLong method will be biased. In the early stage of investigation, researchers don’t always have enough data; therefore, those asymptotic variance estimates such as DeLong’s can be unreliable. In this article we propose a two-way random effects analysis of variance (ANOVA) method to compute an unbiased variance estimate of a two-sample U-statistic of order (1,1) in general, and of the AUC in particular. We...