Abstract
In recent years, wide deployment of automatic face recognition systems has been accompanied by substantial gains in algorithm performance. However, benchmarking tests designed to evaluate these systems do not account for the errors of human operators, who are often an integral part of face recognition solutions in forensic and security settings. This causes a mismatch between evaluation tests and operational accuracy. We address this by measuring user performance in a face recognition system used to screen passport applications for identity fraud. Experiment 1 measured target detection accuracy in algorithm-generated ‘candidate lists’ selected from a large database of passport images. Accuracy was notably poorer than in previous studies of unfamiliar face matching: participants made over 50% errors for adult target faces, and over 60% when matching images of children. Experiment 2 then compared performance of student participants to trained passport officers–who use the system in their daily work–and found equivalent performance in these groups. Encouragingly, a group of highly trained and experienced “facial examiners” outperformed these groups by 20 percentage points. We conclude that human performance curtails accuracy of face recognition systems–potentially reducing benchmark estimates by 50% in operational settings. Mere practise does not attenuate these limits, but superior performance of trained examiners suggests that recruitment and selection of human operators, in combination with effective training and mentorship, can improve the operational accuracy of face recognition systems.
Highlights
In modern forensic and security practice, automatic face recognition software is often used to augment important identification processes
Given the very low overall accuracy in this test, it becomes important to ask whether these levels of performance are observed in people who use Face Recognition (FR) software in their daily work
If error rates in Experiment 1 reflect error in professional operators, realistic estimates of overall accuracy in systems employing both FR algorithms and human operators would be half that reported in benchmark tests of the machine component in FR systems
Summary
In modern forensic and security practice, automatic face recognition software is often used to augment important identification processes. An increasingly common application of face recognition technology is known as one-to-many identification—whereby pattern matching algorithms are used to compare a single probe image to large databases of facial images [1]. This function can be used to protect against identity fraud when issuing national identity documents such as passports, immigration visas and driving licences—by improving detection of duplicate applications by the same individual [2]. In forensic applications, face recognition software enables police officers to use this image evidence to search large databases of known offenders [3, 4, 5]. Similar technology is used to enhance user experience in popular social media platforms
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
