Accuracy, reproducibility, and repeatability of forensic footwear examiner decisions

Abstract

The interpretation of footwear evidence relies on the expertise of forensic footwear examiners. Here we report on the largest study to date of the accuracy, reproducibility (inter-examiner variation), and repeatability (intra-examiner variation) of footwear examiners’ decisions. In this study, 84 practicing footwear examiners each conducted up to 100 comparisons between questioned footwear impressions (provided as photographs and digital images) and known footwear (provided as photographs, transparent test impressions, and digital images), resulting in a total of 6610 comparisons. The quality and characteristics of the impressions were selected to be broadly representative of those encountered in casework. A multilevel conclusion scale was used: 40% of responses were definitive conclusions (identification or exclusion), 14% probable conclusions (high degree of association or indications of non-association), 40% class associations (association of class characteristics or limited association of class characteristics), and 6% neutral conclusions (inconclusive or not suitable). On nonmated comparisons, 0.2% of conclusions were erroneous identifications (false positives), and 1.4% were incorrect responses of “high degree of association.” The majority of erroneous identifications were made by a single participant. On mated comparisons, 6.0% of conclusions were erroneous exclusions (false negatives), and 1.8% were incorrect responses of “indications of non-association.” Erroneous conclusions were sometimes reproduced by different examiners, but rarely repeated by the same examiner—1.1% of erroneous identifications were reproduced (none were repeated) and 19.9% of erroneous exclusions were reproduced (just one was repeated). Examiners’ assessments of whether a questioned impression was suitable for comparison were notably inconsistent and may benefit from standardization. Rates of correct definitive conclusions are directly associated with the quality of the questioned impression and the extent of class similarities/differences between the questioned impression and known footwear.