Combined Use of the Area under the ROC Curve and a measure of Contrast to Evaluate Template Matching Similarity Metrics

Abstract

Template matching is a common method used in medical image analysis to determine how similar regions in a test image are to a given reference image. For the matching, a reference image is moved across a test image and a measure of similarity is computed at each position. This process results in a similarity map, which indicates how well the reference image is matched to the test image at each position. The strength of any template-matching algorithm lies in the similarity metrics used. A novel method is presented for the comparison of the matching performances of various similarity metrics, incorporating the area under the receiver operating characteristic (ROC) curve and a measure of contrast, both computed from the similarity map. The area under the ROC curve is a standard method of evaluating the performance of computer-aided diagnosis systems and provides a measure of how much of the reference image has been correctly matched, with values close to 1 being preferred. The measure of contrast provides an indication of how well the correctly matched regions stand out from the background in the similarity map and values close to 1 are preferred. These two measures are combined to define a quantity known as “matching accuracy”, which ranges from −1 to 1 and is chosen to only be positive when contrast and area under the ROC are both positive. “Matching accuracy” allows quantitative evaluation of the matching performance of similarity metrics and an example of its use, to assess the matching performance of two similarity metrics (Euclidean distance and mutual information), is presented. The similarity metrics are applied to the template matching of pairs of mammographic images and “matching accuracy” is also used to investigate the effect on matching performance of several algorithm parameters (e.g. sampling window size and bit depth).