Abstract
Receiver Operating Characteristic (ROC) surfaces have been studied in the literature essentially during the last decade and are considered as a natural generalization of ROC curves in three-class problems. The volume under the surface (VUS) is useful for evaluating the performance of a trichotomous diagnostic system or a three-class classifier’s overall accuracy when the possible disease condition or sample belongs to one of three ordered categories. In the areas of medical studies and machine learning, the VUS of a new statistical model is typically estimated through a sample of ordinal and continuous measurements obtained by some suitable specimens. However, discrete scales of the prediction are also frequently encountered in practice. To deal with such scenario, in this paper, we proposed a unified and efficient algorithm of linearithmic order, based on dynamic programming, for unbiased estimation of the mean and variance of VUS with unidimensional samples drawn from continuous or non-continuous distributions. Monte Carlo simulations verify our theoretical findings and developed algorithms.
Highlights
After decades of development since early 1950s, receiver operating characteristic (ROC) analysis has found abundant applications in a wide spectrum of scientific and engineering areas [1]–[3], including data mining [4], computer vision [5], [6], signal processing [7]–[10], machine learning [11]–[13], medical decision making [14], psychology [15], and biomedical informatics [16], among others
We produce the three samples X1, X2 and X3 based on Poisson distribution with parameter λ and Geometric distribution with parameter P, denoted by P(λ) and G(P) respectively, in four scenarios, which is listed as follows: Under the four scenarios mentioned above, we evaluate the two methods by exploiting Relative Error of Mean (REM ), which is defined by
COMPARISON OF ALGORITHMS FOR ESTIMATION OF VARIANCE OF Volume Under Ordered Three-Class ROC Surface (VUS) In the following we investigate the capacity of our algorithm for computing the variance of VUS in (29), denoted by V DP, with the algorithm based on graph theory proposed by Waegeman et al [39] and boostrap, a widely used technique in many literature [30], [33], [35], denoted by V GT and V Boostrap respectively, in the aspects of unbiasedness and computational efficiency
Summary
After decades of development since early 1950s, receiver operating characteristic (ROC) analysis has found abundant applications in a wide spectrum of scientific and engineering areas [1]–[3], including data mining [4], computer vision [5], [6], signal processing [7]–[10], machine learning [11]–[13], medical decision making [14], psychology [15], and biomedical informatics [16], among others. Mossman’s boostrap method [30] and the tie breaking used in Liu et al [45] gave contribution in this direction, but the former algorithm is biased and time-consuming, while the latter combined with any existing algorithms based on continuous measurements, e.g. Waegeman et al [39], suffers sever bias (see Section V for more details) Motivated by such unsatisfactory situation, in this paper, we developed a linearithmic algorithm for unbiased estimation of the mean and variance of VUS with three ordi-. Our algorithm is unified, that is, it can simultaneously work for samples drawn from both continuous and discrete populations It is unbiased, in other words, the mean of its output is equivalent to the population version of VUS’s variance, which is always a necessary feature in statistical inference.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
