Presentation Attack Detection Methods Research Articles

Detection of Adversarial Facial Accessory Presentation Attacks Using Local Face Differential

To counter adversarial facial accessory presentation attacks (PAs), a detection method based on local face differential is proposed in this article. It extracts the local face differential features from a suspected face image and a reference face image, and then adaptively fuses the differential features of different local face regions to detect adversarial facial accessory PAs. Meanwhile, the principle of the proposed method is explained by theoretically investigating the local facial differences between a bona fide presentation and an adversarial facial accessory PA when they are compared with a reference face image. To evaluate the proposed method, this article builds a database with different adversarial examples (AEs), presentation attack instruments (PAIs), illumination conditions, and cameras. The experimental results show that it can effectively distinguish between adversarial facial accessory PAs and bona fide presentations, and it has good generalization ability to unseen AEs, PAIs, illumination conditions, and cameras. Moreover, it outperforms the existing AE detection and presentation attack detection methods in detecting adversarial facial accessory PAs.

Multi-Domain Incremental Learning for Face Presentation Attack Detection

Previous face Presentation Attack Detection (PAD) methods aim to improve the effectiveness of cross-domain tasks. However, in real-world scenarios, the original training data of the pre-trained model is not available due to data privacy or other reasons. Under these constraints, general methods for fine-tuning single-target domain data may lose previously learned knowledge, leading to a catastrophic forgetting problem. To address these issues, we propose a multi-domain incremental learning (MDIL) method for PAD, which not only learns knowledge well from the new domain but also maintains the performance of previous domains stably. Specifically, we propose an adaptive domain-specific experts (ADE) framework based on the vision transformer to preserve the discriminability of previous domains. Furthermore, an asymmetric classifier is designed to keep the output distribution of different classifiers consistent, thereby improving the generalization ability. Extensive experiments show that our proposed method achieves state-of-the-art performance compared to prior methods of incremental learning. Excitingly, under more stringent setting conditions, our method approximates or even outperforms the DA/DG-based methods.

Improving Monocular Facial Presentation-Attack-Detection Robustness with Synthetic Noise Augmentations.

We present a synthetic augmentation approach towards improving monocular face presentation-attack-detection (PAD) robustness to real-world noise additions. Face PAD algorithms secure authentication systems against spoofing attacks, such as pictures, videos, and 2D-inspired masks. Best-in-class PAD methods typically use 3D imagery, but these can be expensive. To reduce application cost, there is a growing field investigating monocular algorithms that detect facial artifacts. These approaches work well in laboratory conditions, but can be sensitive to the imaging environment (e.g., sensor noise, dynamic lighting, etc.). The ideal solution for noise robustness is training under all expected conditions; however, this is time consuming and expensive. Instead, we propose that physics-informed noise-augmentations can pragmatically achieve robustness. Our toolbox contains twelve sensor and lighting effect generators. We demonstrate that our toolbox generates more robust PAD features than popular augmentation methods in noisy test-evaluations. We also observe that the toolbox improves accuracy on clean test data, suggesting that it inherently helps discern spoof artifacts from imaging artifacts. We validate this hypothesis through an ablation study, where we remove liveliness pairs (e.g., live or spoof imagery only for participants) to identify how much real data can be replaced with synthetic augmentations. We demonstrate that using these noise augmentations allows us to achieve better test accuracy while only requiring 30% of participants to be fully imaged under all conditions. These findings indicate that synthetic noise augmentations are a great way to improve PAD, addressing noise robustness while simplifying data collection.

Face presentation attack detection: Research opportunities and perspectives

The rapid development of biometric methods and their implementation in practice has led to the widespread attacks called spoofing, which are purely biometric vulnerabilities, but are not used in conjunction with other IT security solutions. Although biometric recognition as a branch of computer science dates back to the 1960s, attacks on biometric systems have become more sophisticated since the 2010s due to great advances in pattern recognition. It should be noted that face recognition is the most attractive topic for deceiving recognition systems. Popular presentation attacks, such as print, replay and mask attacks, have demonstrated a high security risk for SOTA face recognition systems. Many Presentation Attack Detection (PAD) methods (also known as face anti-spoofing methods or countermeasures) have been proposed that can automatically detect and mitigate such targeted attacks. The article presents a systematic survey in face anti-spoofing with prognostic trends in this research area. A brief description of 16 outstanding previous surveys on the face PAD field is mentioned, from which it is possible to trace how this scientific topic has developed. SOTA in PAD provides an analysis of a wide range of the PAD methods, which are categorized into two unbalanced groups: digital (feature-based) and physical (sensor-based) methods. Generalization of deep learning methods as a recent trend aimed at improving recognition results requires special attention. This survey presents five types of generalization such as transfer learning, anomaly detection, few-shot and zero-shot learning, auxiliary supervision, and multi-spectral methods. A summary of over than 40 existing 2D/3D face spoofing databases is a guideline for those who want to select databases for experiments. One can also find a description of performance evaluation metrics and testing protocols. In addition, we discuss trends and perspectives in the emerging field of facial biometrics.

A survey on face presentation attack detection mechanisms: hitherto and future perspectives.

The advances in human face recognition (FR) systems have recorded sublime success for automatic and secured authentication in diverse domains. Although the traditional methods have been overshadowed by face recognition counterpart during this progress, computer vision gains rapid traction, and the modern accomplishments address problems with real-world complexity. However, security threats in FR-based systems are a growing concern that offers a new-fangled track to the research community. In particular, recent past has witnessed ample instances of spoofing attacks where imposter breaches security of the system with an artifact of human face to circumvent the sensor module. Therefore, presentation attack detection (PAD) capabilities are instilled in the system for discriminating genuine and fake traits and anticipation of their impact on the overall behavior of the FR-based systems. To scrutinize exhaustively the current state-of-the-art efforts, provide insights, and identify potential research directions on face PAD mechanisms, this systematic study presents a review of face anti-spoofing techniques that use computational approaches. The study includes advancements in face PAD mechanisms ranging from traditional hardware-based solutions to up-to-date handcrafted features or deep learning-based approaches. We also present an analytical overview of face artifacts, performance protocols, and benchmark face anti-spoofing datasets. In addition, we perform analysis of the twelve recent state-of-the-art (SOTA) face PAD techniques on a common platform using identical dataset (i.e., REPLAY-ATTACK) and performance protocols (i.e., HTER and ACA). Our overall analysis investigates that despite prevalent face PAD mechanisms demonstrate potential performance, there exist some crucial issues that requisite a futuristic attention. Our analysis put forward a number of open issues such as; limited generalization to unknown attacks, inadequacy of face datasets for DL-models, training models with new fakes, efficient DL-enabled face PAD with smaller datasets, and limited discrimination of handcrafted features. Furthermore, the COVID-19 pandemic is an additional challenge to the existing face-based recognition systems, and hence to the PAD methods. Our motive is to present a complete reference of studies in this field and orient researchers to promising directions.

Presentation attack detection and biometric recognition in a challenge-response formalism

Presentation attack detection (PAD) is used to mitigate the dangers of the weakest link problem in biometric recognition, in which failure modes of one application affect the security of all other applications. Strong PAD methods are therefore a must, and we believe biometric challenge-response protocols (BCRP) form an underestimated part of this ecosystem. In this paper, we conceptualize what BCRPs are, and we propose a descriptive formalism and categorization for working with them. We validate the categorization against existing literature that we classified to be describing BCRPs. Lastly, we discuss how strong BCRPs provide advantages over PAD methods, specifically in the protection of individual applications and the protection of other applications from inadvertent leaks in BCRP applications. We note that research in BCRPs is fragmented, and our intent for the proposed formalism and categorization are to give focus and direction to research efforts into biometric challenge-response protocols.

Presentation attack detection based on two-stream vision transformers with self-attention fusion

Aiming at the performance degradation of the existing presentation attack detection methods due to the illumination variation, a two-stream vision transformers framework (TSViT) based on transfer learning in two complementary spaces is proposed in this paper. The face images of RGB color space and multi-scale retinex with color restoration (MSRCR) space are fed to TSViT to learn the distinguishing features of presentation attack detection. To effectively fuse features from two sources (RGB color space images and MSRCR images), a feature fusion method based on self-attention is built, which can effectively capture the complementarity of two features. Experiments and analysis on Oulu-NPU, CASIA-MFSD, and Replay-Attack databases show that it outperforms most existing methods in intra-database testing and achieves good generalization performance in cross-database testing.

Fingerprint Presentation Attack Detection by Channel-Wise Feature Denoising

Due to the diversity of attack materials, fingerprint recognition systems (AFRSs) are vulnerable to malicious attacks. It is thus important to propose effective fingerprint presentation attack detection (PAD) methods for the safety and reliability of AFRSs. However, current PAD methods often exhibit poor robustness under new attack types settings. This paper thus proposes a novel channel-wise feature denoising fingerprint PAD (CFD-PAD) method by handling the redundant noise information ignored in previous studies. The proposed method learns important features of fingerprint images by weighing the importance of each channel and identifying discriminative channels and "noise" channels. Then, the propagation of "noise" channels is suppressed in the feature map to reduce interference. Specifically, a PA-Adaptation loss is designed to constrain the feature distribution to make the feature distribution of live fingerprints more aggregate and that of spoof fingerprints more disperse. Experimental results evaluated on the LivDet 2017 dataset showed that the proposed CFD-PAD can achieve a 2.53% average classification error (ACE) and a 93.83% true detection rate when the false detection rate equals 1.0% (TDR@FDR=1%). Also, the proposed method markedly outperforms the best single-model-based methods in terms of ACE (2.53% vs. 4.56%) and TDR@FDR=1%(93.83% vs. 73.32%), which demonstrates its effectiveness. Although we have achieved a comparable result with the state-of-the-art multiple-model-based methods, there still is an increase in TDR@FDR=1% from 91.19% to 93.83%. In addition, the proposed model is simpler, lighter and more efficient and has achieved a 74.76% reduction in computation time compared with the state-of-the-art multiple-model-based method. The source code is available at https://github.com/kongzhecn/cfd-pad.

Domain-Specific Adaptation of CNN for Detecting Face Presentation Attacks in NIR

For the automotive industry moving towards personalized applications and experiences, the identification of the person inside vehicle is necessary; and it must be carried out in a secure manner. In this paper, we propose a unique face presentation attack detection (PAD) system for operation inside a passenger vehicle. A typical <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-vehicular</i> face PAD system is required to function with several constraints such as bounded sensing (imaging) capabilities, limited computing resources on embedded devices, real-time inference, and essentially, very high accuracy. In this work, we develop a face PAD system for automotive domain, relying on a single NIR camera, to continually verify whether the driver’s face is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bona-fide</i> or not. Our work has two main contributions: first, a lightweight face PAD framework has been developed using a 9-layer convolutional neural network (CNN). With its compact size and limited set of operators, it can be deployed in a resource constrained embedded device to achieve a near real-time inference. To alleviate the problem of limited training data (face PAD in NIR) for a given system, we develop an efficient mechanism to obtain this CNN through the combination of adaptation of domain-specific layers and task-specific fine-tuning of a base CNN. As the second contribution, we collect a large face PAD dataset with 5800+ videos, acquired in NIR (940 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nm</i> ) illumination, for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-vehicular</i> use-cases. This dataset, named VFPAD, captures several real-world variations in terms of environmental settings, illumination, subject’s pose, and appearances. Based on the VFPAD dataset, we demonstrate that the proposed face PAD method achieves very high performance (overall accuracy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx 98.0$ </tex-math></inline-formula> %), and also outperforms several baseline face PAD methods. The dataset will be shared with the wider scientific community for research purposes.

Recent advancements in finger vein recognition technology: Methodology, challenges and opportunities

Finger vein recognition biometric trait is a significant biometric modality that is considered more secure, reliable, and emerging. This article presents a review to focus on the recent research landscape in biometric finger vein recognition systems. This article focuses on manuscripts related to keywords ‘Finger Vein Authentication System’, ‘Anti-spoofing or Presentation Attack Detection’, ‘Multimodal Biometric Finger Vein Authentication’ and their variations in four main digital research libraries such as IEEE Xplore, Springer, ACM, and Science Direct. The final set of articles is divided into three main categories: Deep Learning (DL) based finger vein recognition, Presentation Attack Detection (PAD), and Multimodal-based finger vein authentication system. Deep learning-based finger vein recognition techniques are further sub-divided into pre-processing (Quality assessment and enhancement) based, feature extraction based, and feature extraction and recognition based schemes. Presentation attack detection methods are sub-divided into software-based and hardware-based approaches. Multimodal-based finger vein biometric system is sub-categorized into feature level fusion, matching level fusion, and hybrid fusion methods. The authors have studied the problem of the recent algorithm and their solution related to finger vein biometric system from the recent literature. Performance analysis and selected the best promising research work from the mentioned studies are also presented. Finally, open challenges, opportunities, and suggested solutions related to deep learning, PAD, and Multimodal based finger vein recognition systems have been discussed in the discussion section. This work would be helpful to the developers, company users, researchers, and readers to get insight into the importance of such technology and the recent problem faced by finger vein authentication technology with respect to deep learning and multimodal systems.

A Study of Gender Bias in Face Presentation Attack and Its Mitigation

In biometric systems, the process of identifying or verifying people using facial data must be highly accurate to ensure a high level of security and credibility. Many researchers investigated the fairness of face recognition systems and reported demographic bias. However, there was not much study on face presentation attack detection technology (PAD) in terms of bias. This research sheds light on bias in face spoofing detection by implementing two phases. First, two CNN (convolutional neural network)-based presentation attack detection models, ResNet50 and VGG16 were used to evaluate the fairness of detecting imposer attacks on the basis of gender. In addition, different sizes of Spoof in the Wild (SiW) testing and training data were used in the first phase to study the effect of gender distribution on the models’ performance. Second, the debiasing variational autoencoder (DB-VAE) (Amini, A., et al., Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure) was applied in combination with VGG16 to assess its ability to mitigate bias in presentation attack detection. Our experiments exposed minor gender bias in CNN-based presentation attack detection methods. In addition, it was proven that imbalance in training and testing data does not necessarily lead to gender bias in the model’s performance. Results proved that the DB-VAE approach (Amini, A., et al., Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure) succeeded in mitigating bias in detecting spoof faces.

An exploratory study of interpretability for face presentation attack detection

Biometric recognition and presentation attack detection (PAD) methods strongly rely on deep learning algorithms. Though often more accurate, these models operate as complex black boxes. Interpretability tools are now being used to delve deeper into the operation of these methods, which is why this work advocates their integration in the PAD scenario. Building upon previous work, a face PAD model based on convolutional neural networks was implemented and evaluated both through traditional PAD metrics and with interpretability tools. An evaluation on the stability of the explanations obtained from testing models with attacks known and unknown in the learning step is made. To overcome the limitations of direct comparison, a suitable representation of the explanations is constructed to quantify how much two explanations differ from each other. From the point of view of interpretability, the results obtained in intra and inter class comparisons led to the conclusion that the presence of more attacks during training has a positive effect in the generalisation and robustness of the models. This is an exploratory study that confirms the urge to establish new approaches in biometrics that incorporate interpretability tools. Moreover, there is a need for methodologies to assess and compare the quality of explanations.

A review of state-of-the-art in Face Presentation Attack Detection: From early development to advanced deep learning and multi-modal fusion methods

Face Recognition is considered one of the most common biometric solutions these days and is widely used across a range of devices for various security purposes. The performance of FR systems has improved by orders of magnitude over the past decade. This is mainly due to the latest developments in computer vision and deep convolutional neural networks, and the availability of large training datasets. At the same time, these systems have been subject to various types of attacks. Presentation attacks are common, simple, and easy to implement. These simply involve presenting a video, photo, or mask to the camera or digital sensor and have proven capable of fooling FR systems and providing access to unauthorised users. Presentation attack detection is increasingly attracting more attention in the research community. A wide range of methods has already been developed to address this challenge. Deep learning-based methods in particular have shown very promising results. However, existing literature suggests that even with state-of-the-art methods, performance drops significantly in cross-dataset evaluation. We present a thorough, comprehensive, and technical review of existing literature on this timely and challenging problem. We first introduce and discuss the presentation attack problem and cover related and recent work in this area. In-depth technical details of existing presentation attack detection methods are then presented and critically discussed and evaluated, followed by a comprehensive discussion and evaluation of existing public datasets and commonly used evaluation metrics. Our review shows clearly that despite the recent and significant advances in this area of research, detecting unseen attacks is still considered a key problem. Machine learning methods tend to perform well, but only when test data comes from the same distribution as the training data (i.e. same dataset). New research directions are discussed in detail, including ways to improve the generalisation of machine learning methods, and move towards creating more stable presentation attack detection techniques that generalise across a wide range of unseen samples.

Anomaly Detection With Convolutional Autoencoders for Fingerprint Presentation Attack Detection

In recent years, the popularity of fingerprint-based biometric authentication systems significantly increased. However, together with many advantages, biometric systems are still vulnerable to presentation attacks (PAs). In particular, this applies for unsupervised applications, where new attacks unknown to the system operator may occur. Therefore, presentation attack detection (PAD) methods are used to determine whether samples stem from a bona fide subject or from a presentation attack instrument (PAI). In this context, most works are dedicated to solve PAD as a two-class classification problem, which includes training a model on both bona fide and PA samples. In spite of the good detection rates reported, these methods still face difficulties detecting PAIs from unknown materials. To address this issue, we propose a new PAD technique based on autoencoders (AEs) trained only on bona fide samples (i.e. one-class), which are captured in the short wave infrared domain. On the experimental evaluation over a database of 19,711 bona fide and 4,339 PA images including 45 different PAI species, a detection equal error rate (D-EER) of 2.00% was achieved. Additionally, our best performing AE model is compared to further one-class classifiers (support vector machine, Gaussian mixture model). The results show the effectiveness of the AE model as it significantly outperforms the previously proposed methods.

On the generalisation capabilities of fingerprint presentation attack detection methods in the short wave infrared domain

Nowadays, fingerprint-based biometric recognition systems are becoming increasingly popular. However, in spite of their numerous advantages, biometric capture devices are usually exposed to the public and thus vulnerable to presentation attacks (PAs). Therefore, presentation attack detection (PAD) methods are of utmost importance in order to distinguish between bona fide and attack presentations. Owing to the nearly unlimited possibilities to create new presentation attack instruments (PAIs), unknown attacks are a threat to the existing PAD algorithms. This fact motivates research on generalisation capabilities in order to find PAD methods that are resilient to new attacks. In this context, the authors evaluate the generalisability of multiple PAD algorithms on a dataset of 19,711 bona fide and 4339 PA samples, including 45 different PAI species. The PAD data is captured in the short wave infrared domain, and the results discuss the advantages and drawbacks of this PAD technique regarding unknown attacks.

Latent regularized generative adversarial network for face spoofing detection

Face anti-spoofing technology is critical to prevent face recognition systems from experiencing a security breach. Most of presentation attack detection (PAD) methods consider the task as a supervised binary classification problem. Many of these methods struggle to grasp adequate spoofing cues and generalize poorly. In this paper, we formulate the face anti-spoofing detection as an anomaly detection task to tackle the generalization issue. A novel deep network is proposed by using adversarial training under semi-supervised learning framework. The underlying structure of training data is captured in the image reconstruction space and can be further restricted in the space of latent representation in a discriminant manner, leading to a more robust spoof detector. In the test, the attacks are regarded as out-of-distributions samples that naturally exhibit a higher feature reconstruction error in the latent space than real samples in the dataset. Experiments show that our model is clearly superior over cutting-edge semi-supervised abnormal detectors and achieves state-of-the-art results on both intra- and inter-database testing.

Local feature encoding for unknown presentation attack detection: An analysis of different local feature descriptors

In spite of the advantages of using fingerprints for subject authentication, several works have shown that fingerprint recognition systems can be easily circumvented by means of artificial fingerprints or presentation attack instruments (PAIs). In order to address that threat, the existing presentation attack detection (PAD) methods have reported a high detection performance when materials used for the fabrication of PAIs and capture devices are known. However, for more complex and realistic scenarios where one of those factors remains unknown, these PAD methods are unable to correctly separate a PAI from a real fingerprint (i.e. bona fide presentation). In this article, a new PAD approach based on the Fisher Vector technique, which combines local and global information of several local feature descriptors in order to improve the PAD generalisation capabilities, was proposed. The experimental results over unknown scenarios taken from LivDet 2011 to LivDet 2017 show that our proposal reduces the top state-of-the-art average classification error rates by up to four times, thereby making it suitable in real applications demanding high security. In addition, the best single configuration achieved the best results in the LivDet 2019 competition, with an overall accuracy of 96.17%.

Passive imaging at 250 GHz for detection of face presentation attacks.

Face presentation attacks are becoming more efficient since new 3D facial masks are used. Passive terahertz imaging offers specific physical properties that may improve presentation attack detection capabilities. The non-zero transmission capability through a variety of non-metallic materials may provide necessary information for presentation attack detection. The aim of this paper is to present outcomes of a study on face presentation attack detection using passive imaging at 250 GHz. An analysis of presentation attacks for facial recognition systems using custom displayed and printed photographs, 3D-printed and full-face flexible 3D-latex masks, is provided together with spectral characterization of various presentation attack instruments. A set of experiments with various instruments and various sets of clothing is described and discussed. Finally, two presentation attack detection methods are proposed. The first method is based on a threshold corresponding to a difference between mean intensities of selected regions of interests while the second method uses eight different deep learning classifiers to detect presentation attacks. Results of two validation schemes are presented.

Fingerprint Presentation Attack Detection Based on Local Features Encoding for Unknown Attacks

Fingerprint-based biometric systems have experienced a large development in the past. In spite of many advantages, they are still vulnerable to attack presentations (APs). Therefore, the task of determining whether a sample stems from a live subject (i.e., bona fide) or from an artificial replica is a mandatory requirement which has recently received a considerable attention. Nowadays, when the materials for the fabrication of the Presentation Attack Instruments (PAIs) have been used to train the Presentation Attack Detection (PAD) methods, the PAIs can be successfully identified in most cases. However, current PAD methods still face difficulties detecting PAIs built from unknown materials and/or unknown recepies, or acquired using different capture devices. To tackle this issue, we propose a new PAD technique based on three image representation approaches combining local and global information of the fingerprint. By transforming these representations into a common feature space, we can correctly discriminate bona fide from attack presentations in the aforementioned scenarios. The experimental evaluation of our proposal over the LivDet 2011 to 2019 databases, yielded error rates outperforming the top state-of-the-art results by up to 72% in the most challenging scenarios. In addition, the best representation achieved the best results in the LivDet 2019 competition (overall accuracy of 96.17%).

One-Class Fingerprint Presentation Attack Detection Using Auto-Encoder Network.

Automated Fingerprint Recognition Systems (AFRSs) have been threatened by Presentation Attack (PA) since its existence. It is thus desirable to develop effective presentation attack detection (PAD) methods. However, the unpredictable PAs make PAD be a challenging problem. This paper proposes a novel One-Class PAD (OCPAD) method for Optical Coherence Technology (OCT) images based fingerprint PA detection. The proposed OCPAD model is learned from a training set only consists of Bonafides (i.e. real fingerprints). The reconstruction error and latent code obtained from the trained auto-encoder network in the proposed model is taken as the basis for the following spoofness score calculation. To get more accurate reconstruction error, we propose an activation map based weighting model to further refine the accuracy of reconstruction error. We test different statistics and distance measures and finally use a decision level fusion to make the final prediction. Our experiments are performed using a dataset with 93200 bonafide scans and 48400 PA scans. The results show that the proposed OCPAD can achieve a True Positive Rate (TPR) of 99.43% when the False Positive Rate (FPR) equals to 10% and a TPR of 96.59% when FPR=5%, which significantly outperformed a feature based approach and a supervised learning based model requiring PAs for training.