Repeatability and reproducibility of forensic likelihood ratio methods when sample size ratio varies

Abstract

Existing statistical methods for estimating the log-likelihood ratio from biometric scores include parametric estimation, kernel density estimation, and recently adopted logistic regression estimation. There has been a growing interest to study the repeatability and reproducibility of these methods on biometric datasets after the 2009 National Research Council report [15] and the 2016 President's Council of Advisors on Science and Technology report [1]. For a statistical forensic evaluation method to be repeatable, it needs to generate consistent log-likelihood ratios for various sample size ratios between the genuine (mated) and imposter (non-mated) scores from the same database. It is a well known fact, that for logistic regression methods, the estimated intercept value depends on the sample size ratio between the two groups. Therefore, when computing log-likelihood ratios using logistic regression estimation, different genuine and impostor sample size ratios could result in different log-likelihood ratio values. We performed extensive simulations and used face and fingerprint biometric datasets to investigate repeatability and reproducibility of existing log-likelihood ratio estimation methods.