Validation of a feature-based likelihood ratio method for the SAILR software. Part II: Elemental compositional data for comparison of glass samples

Abstract

SAILR is open-source software designed to calculate forensic likelihood ratios (LR) from probability distributions of reference data. The purpose of this study was to demonstrate validation of a multivariate feature-based LR method for SAILR using compositional data on glass fragments. Validation was performed using designated performance characteristics, e.g., accuracy, discrimination, and calibration. These characteristics were measured using performance metrics such as cost of the log likelihood ratio and equal error rate. The LR method was developed simultaneously to a baseline method having features less discriminating, but being better aligned with the normality assumption for within-source variation. The baseline method served as the floor of acceptable performance. The results showed that the available data supported LR methods using three elemental features or less. Best performance was obtained using calcium, magnesium, and silicon. The within-source variation in elemental features was slightly leptokurtic (heavy-tailed), violating the assumption of normality. The data were therefore normalized using Lambert W transformation and the performance of the LR method using normalized data was compared with that using non-normalized data. Although performance improved with normalization, the difference was small. Limits of LR output were set to 1/512 ≤ LR ≤ 158 using the empirical lower and upper boundaries (ELUB) LR method. This limited range was primarily a consequence of notable within-source variation. By passing the tests of normality and outperforming the baseline method, the method was considered valid for use in SAILR for data relevant to the background data set, using the defined range of LRs.