Vulnerability Of Face Recognition Systems Research Articles

Face Anti-Spoofing Detection with Multi-Modal CNN Enhanced by ResNet

The growing prevalence of face recognition technology in various applications, including mobile devices, access control, and financial transactions, highlights its importance. However, the vulnerability of face recognition systems to attacks has been demonstrated, underscoring the necessity of addressing potential weaknesses that attackers may exploit. The paper delves into face presentation attack detection (PAD) within biometric systems, which is crucial for ensuring the reliability and security of face recognition algorithms. To address this issue, the paper proposes a method for face presentation attack detection using ResNet-50 in conjunction with multi-modal data, incorporating RGB, depth, infrared (IR), and thermal channels. The method explores diverse strategies to combine results from each modality, investigating various fusion techniques such as majority voting, weighted voting, average pooling, and a stacking classifier. The system has been tested on the WMCA dataset. It exhibits strong performance compared to existing methods, notably achieving an impressive ACER ratio of 0.087% with the stacking classifier. This approach proves effective by consolidating multiple modalities without requiring individual scenario-specific models, indicating promise for real-world applications

Towards minimizing efforts for Morphing Attacks-Deep embeddings for morphing pair selection and improved Morphing Attack Detection.

Face Morphing Attacks pose a threat to the security of identity documents, especially with respect to a subsequent access control process, because they allow both involved individuals to use the same document. Several algorithms are currently being developed to detect Morphing Attacks, often requiring large data sets of morphed face images for training. In the present study, face embeddings are used for two different purposes: first, to pre-select images for the subsequent large-scale generation of Morphing Attacks, and second, to detect potential Morphing Attacks. Previous studies have demonstrated the power of embeddings in both use cases. However, we aim to build on these studies by adding the more powerful MagFace model to both use cases, and by performing comprehensive analyses of the role of embeddings in pre-selection and attack detection in terms of the vulnerability of face recognition systems and attack detection algorithms. In particular, we use recent developments to assess the attack potential, but also investigate the influence of morphing algorithms. For the first objective, an algorithm is developed that pairs individuals based on the similarity of their face embeddings. Different state-of-the-art face recognition systems are used to extract embeddings in order to pre-select the face images and different morphing algorithms are used to fuse the face images. The attack potential of the differently generated morphed face images will be quantified to compare the usability of the embeddings for automatically generating a large number of successful Morphing Attacks. For the second objective, we compare the performance of the embeddings of two state-of-the-art face recognition systems with respect to their ability to detect morphed face images. Our results demonstrate that ArcFace and MagFace provide valuable face embeddings for image pre-selection. Various open-source and commercial-off-the-shelf face recognition systems are vulnerable to the generated Morphing Attacks, and their vulnerability increases when image pre-selection is based on embeddings compared to random pairing. In particular, landmark-based closed-source morphing algorithms generate attacks that pose a high risk to any tested face recognition system. Remarkably, more accurate face recognition systems show a higher vulnerability to Morphing Attacks. Among the systems tested, commercial-off-the-shelf systems were the most vulnerable to Morphing Attacks. In addition, MagFace embeddings stand out as a robust alternative for detecting morphed face images compared to the previously used ArcFace embeddings. The results endorse the benefits of face embeddings for more effective image pre-selection for face morphing and for more accurate detection of morphed face images, as demonstrated by extensive analysis of various designed attacks. The MagFace model is a powerful alternative to the often-used ArcFace model in detecting attacks and can increase performance depending on the use case. It also highlights the usability of embeddings to generate large-scale morphed face databases for various purposes, such as training Morphing Attack Detection algorithms as a countermeasure against attacks.

Comprehensive Vulnerability Evaluation of Face Recognition Systems to Template Inversion Attacks Via 3DFace Reconstruction.

In this paper, we comprehensively evaluate the vulnerability of state-of-the-art face recognition systems to template inversion attacks using 3D face reconstruction. We propose a new method (called GaFaR) to reconstruct 3D faces from facial templates using a pretrained geometry-aware face generation network, and train a mapping from facial templates to the intermediate latent space of the face generator network. We train our mapping with a semi-supervised approach using real and synthetic face images. For real face images, we use a generative adversarial network (GAN)-based framework to learn the distribution of generator intermediate latent space. For synthetic face images, we directly learn the mapping from facial templates to the generator intermediate latent code. Furthermore, to improve the success attack rate, we use two optimization methods on the camera parameters of the GNeRF model. We propose our method in the whitebox and blackbox attacks against face recognition systems and compare the transferability of our attack with state-of-the-art methods across other face recognition systems on the MOBIO and LFW datasets. We also perform practical presentation attacks on face recognition systems using the digital screen replay and printed photographs, and evaluate the vulnerability of face recognition systems to different template inversion attacks. The project page is available at https://www.idiap.ch/paper/gafar.

Psychophysical Evaluation of Human Performance in Detecting Digital Face Image Manipulations

In recent years, increasing deployment of face recognition technology in security-critical settings, such as border control or law enforcement, has led to considerable interest in the vulnerability of face recognition systems to attacks utilising legitimate documents, which are issued on the basis of digitally manipulated face images. As automated manipulation and attack detection remains a challenging task, conventional processes with human inspectors performing identity verification remain indispensable. These circumstances merit a closer investigation of human capabilities in detecting manipulated face images, as previous work in this field is sparse and often concentrated only on specific scenarios and biometric characteristics. This work introduces a web-based, remote visual discrimination experiment on the basis of principles adopted from the field of psychophysics and subsequently discusses interdisciplinary opportunities with the aim of examining human proficiency in detecting different types of digitally manipulated face images, specifically face swapping, morphing, and retouching. In addition to analysing appropriate performance measures, a possible metric of detectability is explored. Experimental data of 306 probands indicate that detection performance is widely distributed across the population and detection of certain types of face image manipulations is much more challenging than others.

Face Morphing Attack Generation and Detection: A Comprehensive Survey

Face recognition has been successfully deployed in real-time applications, including secure applications such as border control. The vulnerability of face recognition systems (FRSs) to various kinds of attacks (both direct and indirect attacks) and face morphing attacks has received great interest from the biometric community. The goal of a morphing attack is to subvert an FRS at an automatic border control (ABC) gate by presenting an electronic machine-readable travel document (eMRTD) or e-passport that is obtained based on a morphed face image. Since the application process for an e-passport in the majority of countries requires a passport photograph to be presented by the applicant, a malicious actor and an accomplice can generate a morphed face image to obtain the e-passport. An e-passport with a morphed face image can be used by both the malicious actor and the accomplice to cross a border, as the morphed face image can be verified against both of them. This can result in a significant threat, as a malicious actor can cross the border without revealing the trace of his/her criminal background, while the details of the accomplice are recorded in the log of the access control system. This survey aims to present a systematic overview of the progress made in the area of face morphing in terms of both morph generation and morph detection. In this article, we describe and illustrate various aspects of face morphing attacks, including different techniques for generating morphed face images and state-of-the-art morph attack detection (MAD) algorithms based on a stringent taxonomy as well as the availability of public databases, which allow us to benchmark new MAD algorithms in a reproducible manner. The outcomes of competitions and benchmarking, vulnerability assessments, and performance evaluation metrics are also provided in a comprehensive manner. Furthermore, we discuss the open challenges and potential future areas that need to be addressed in the evolving field of biometrics.

Face morph detection for unknown morphing algorithms and image sources: a multi‐scale block local binary pattern fusion approach

The vulnerability of face recognition systems against so-called morphing attacks has been revealed in the past years. Recently, different kinds of morphing attack detection approaches have been proposed. However, the vast majority of published results has been obtained from rather constrained experimental setups. In particular, most investigations do not consider variations in morphing techniques, image sources, and image post-processing. Hence, reported performance rates can not be maintained in realistic scenarios, as the NIST FRVT MORPH performance evaluation showed. In this work, existing algorithms are benchmarked on a new, more realistic database. This database consists of two different data sets, from which morphs were created using four different morphing algorithms. In addition, the database contains four different post-processings (including print-scan transformation and JPEG2000 compression). Further, a new morphing attack detection method based on a fusion of different configurations of Multi-scale Block Local Binary Patterns (MB-LBP) on an image divided into multiple cells is presented. The proposed score-level fusion of a maximum number of 18 different configurations is shown to significantly improve the robustness of the resulting morphing attack detection scheme, yielding an average performance between 2.26% and 8.52% in terms of Detection Equal Error Rate (D-EER), depending on the applied post-processing.

Face presentation attack detection in mobile scenarios: A comprehensive evaluation

The vulnerability of face recognition systems to different presentation attacks has aroused increasing concern in the biometric community. Face presentation detection (PAD) techniques, which aim to distinguish real face samples from spoof artifacts, are the efficient countermeasure. In recent years, various methods have been proposed to address 2D type face presentation attacks, including photo print attack and video replay attack. However, it is difficult to tell which methods perform better for these attacks, especially in practical mobile authentication scenarios, since there is no systematic evaluation or benchmark of the state-of-the-art methods on a common ground (i.e., using the same databases and protocols). Therefore, this paper presents a comprehensive evaluation of several representative face PAD methods (30 in total) on three public mobile spoofing datasets to quantitatively compare the detection performance. Furthermore, the generalization ability of existing methods is tested under cross-database testing scenarios to show the possible database bias. We also summarize meaningful observations and give some insights that will help promote both academic research and practical applications.

Light Field-Based Face Presentation Attack Detection: Reviewing, Benchmarking and One Step Further

Vulnerability of face recognition systems to presentation attacks has attracted increasing attention from the biometrics and forensics communities. Moreover, the recent availability of light field cameras is opening new possibilities for designing improved face presentation attack detection solutions. In this context, this paper provides the first review and benchmarking study in the literature on light field-based face presentation attack detection solutions. State-of-the-art solutions are assessed in terms of accuracy, generalization and complexity, using a common, representative evaluation framework. This paper also proposes a novel face presentation attack detection solution, based on a histogram of oriented gradients descriptor, which exploits the disparity information available in light field imaging. The evaluation of the proposed face presentation attack detection solution for different presentation attack types shows a very effective and stable performance, notably performing better than the state-of-the-art alternatives.

Presentation Attack Detection Methods for Face Recognition Systems

The vulnerability of face recognition systems to presentation attacks (also known as direct attacks or spoof attacks) has received a great deal of interest from the biometric community. The rapid evolution of face recognition systems into real-time applications has raised new concerns about their ability to resist presentation attacks, particularly in unattended application scenarios such as automated border control. The goal of a presentation attack is to subvert the face recognition system by presenting a facial biometric artifact. Popular face biometric artifacts include a printed photo, the electronic display of a facial photo, replaying video using an electronic display, and 3D face masks. These have demonstrated a high security risk for state-of-the-art face recognition systems. However, several presentation attack detection (PAD) algorithms (also known as countermeasures or antispoofing methods) have been proposed that can automatically detect and mitigate such targeted attacks. The goal of this survey is to present a systematic overview of the existing work on face presentation attack detection that has been carried out. This paper describes the various aspects of face presentation attacks, including different types of face artifacts, state-of-the-art PAD algorithms and an overview of the respective research labs working in this domain, vulnerability assessments and performance evaluation metrics, the outcomes of competitions, the availability of public databases for benchmarking new PAD algorithms in a reproducible manner, and finally a summary of the relevant international standardization in this field. Furthermore, we discuss the open challenges and future work that need to be addressed in this evolving field of biometrics.

Presentation attack detection for face recognition using light field camera.

The vulnerability of face recognition systems isa growing concern that has drawn the interest from both academic and research communities. Despite the availability of a broad range of face presentation attack detection (PAD)(or countermeasure or antispoofing) schemes, there exists no superior PAD technique due to evolution of sophisticated presentation attacks (or spoof attacks). In this paper, we present a new perspective for face presentation attack detection by introducing light field camera (LFC). Since the use of a LFC can record the direction of each incoming ray in addition to the intensity, it exhibits an unique characteristic of rendering multiple depth(or focus) images in a single capture. Thus, we present a novel approach that involves exploring the variation of the focus between multiple depth (or focus) images rendered by the LFC that in turn can be used to reveal the presentation attacks. To this extent, we first collect a new face artefact database using LFC that comprises of 80 subjects. Face artefacts are generated by simulating two widely used attacks, such as photo print and electronic screen attack. Extensive experiments carried out on the light field face artefact database have revealed the outstanding performance of the proposed PAD scheme when benchmarked with various well established state-of-the-art schemes.