Virtual image standard (VIS) for performance evaluation of the congruent matching cells (CMC) algorithms in firearm evidence identifications

Abstract

The congruent matching cells (CMC) method was invented at the National Institute of Standards and Technology (NIST) in 2012 for automatic and objective firearm evidence identifications and estimation of the weight of evidence in firearm evidence identifications. Since 2013, five CMC algorithms have been developed at NIST. In this paper, the virtual image standard (VIS) is proposed through trimming and stitching KNM images for quantitative performance evaluations of different CMC algorithms. The evaluation criteria include the correlation accuracy (both the CMC numbers and distribution pattern), correlation efficiency, false positive (FP) error rate, and the maximum separation of known matching (KM) and known non-matching (KNM) image pairs. The VIS composes correlation cells from different KNM images, which can provide a ground truth for verifying the CMC numbers, distribution patterns, and FP errors. By identifying three groups of VIS, the Convergence CMC algorithm showed superior performances for the future casework in firearm evidence identifications. Lastly, the success of this study suggests that the VIS could also be used to optimize the correlation parameters, to develop and test new CMC algorithms, and evaluate the performance before it is put into use for firearm examiner's casework.