Weighted performance index for objective evaluation of border detection methods in dermoscopy images

Abstract

This paper presents a novel approach for objective evaluation of border detection in dermoscopy images of melanoma. In melanoma studies, border detection is a fundamental step toward the development of a computer-aided diagnosis system. Therefore, its accuracy is essential for accurate implementation of the subsequent parts of the diagnostic system. An objective evaluation procedure of border detection methods is presented. The evaluation procedure uses the weighted performance index, which is composed of weighted metrics of sensitivity, specificity, accuracy, precision, border error and similarity. This index can also be used to optimize the parameters of a border detection method. Experiments are performed on 55 high-resolution dermoscopy images. Using the union of four sets of dermatologist-drawn borders as the ground truth, weighted metrics of sensitivity, specificity, accuracy, precision, border error and similarity are evaluated. Then, the weighted performance index is constructed and used to optimize the parameters of the hybrid border detection method. The outcome of the optimization process, verified through statistical analysis, yields a higher degree of agreement between automatic borders and the ground truth, compared with using standard metrics only. Finally, the weighted performance index is used to evaluate five recently reported border detection methods.