On the reproducibility and repeatability of likelihood ratio in forensics: A case study using face biometrics

Abstract

When using biometric technology in forensic applications, it is necessary to compute a Log-likelihood Ratio (LLR) for a given piece of evidence (E) under two competing hypotheses, namely the prosecution and the defence hypotheses. Although LLR is a quantity expressing uncertainty and intuitively quantifying its uncertainty would not make sense, in practice, it is computed under a set of assumptions and methods for a given data set. Therefore, it is essential to ask how well and how repeatable and/or reproducible it is that we can estimate LLR. More specifically, it is desirable to understand the behaviour of the confidence intervals of the estimated LLR for any feasible region since any incorrect estimate may lead to possible condemnation of innocent people. To this end, we have thus tackled the estimate of LLR which is fundamentally a Bayesian concept using a frequentist approach, via bootstraping, using two LLR estimators, namely Logistic Regression (LR) and Kernel Density Estimator (KDE). The experimental results, which are based on seven face recognition systems, show that LLR does have different confidence lengths, thus highlighting that LLR cannot be estimated with the same certainty everywhere. Moreover, for the two LLR estimators investigated, we found that there is a consistent region in which any LLR value can be estimated confidently. To our best knowledge, these two findings have never been systematically reported in literature. They thus advance our understanding of LLR when used in computing the strength of biometric evidence in forensics.