Face Presentation Attack Detection Research Articles

Multi-Domain Incremental Learning for Face Presentation Attack Detection

Previous face Presentation Attack Detection (PAD) methods aim to improve the effectiveness of cross-domain tasks. However, in real-world scenarios, the original training data of the pre-trained model is not available due to data privacy or other reasons. Under these constraints, general methods for fine-tuning single-target domain data may lose previously learned knowledge, leading to a catastrophic forgetting problem. To address these issues, we propose a multi-domain incremental learning (MDIL) method for PAD, which not only learns knowledge well from the new domain but also maintains the performance of previous domains stably. Specifically, we propose an adaptive domain-specific experts (ADE) framework based on the vision transformer to preserve the discriminability of previous domains. Furthermore, an asymmetric classifier is designed to keep the output distribution of different classifiers consistent, thereby improving the generalization ability. Extensive experiments show that our proposed method achieves state-of-the-art performance compared to prior methods of incremental learning. Excitingly, under more stringent setting conditions, our method approximates or even outperforms the DA/DG-based methods.

Federated Generalized Face Presentation Attack Detection.

Face presentation attack detection (fPAD) plays a critical role in the modern face recognition pipeline. An fPAD model with good generalization can be obtained when it is trained with face images from different input distributions and different types of spoof attacks. In reality, training data (both real face images and spoof images) are not directly shared between data owners due to legal and privacy issues. In this article, with the motivation of circumventing this challenge, we propose a federated face presentation attack detection (FedPAD) framework that simultaneously takes advantage of rich fPAD information available at different data owners while preserving data privacy. In the proposed framework, each data owner (referred to as data centers) locally trains its own fPAD model. A server learns a global fPAD model by iteratively aggregating model updates from all data centers without accessing private data in each of them. Once the learned global model converges, it is used for fPAD inference. To equip the aggregated fPAD model in the server with better generalization ability to unseen attacks from users, following the basic idea of FedPAD, we further propose a federated generalized face presentation attack detection (FedGPAD) framework. A federated domain disentanglement strategy is introduced in FedGPAD, which treats each data center as one domain and decomposes the fPAD model into domain-invariant and domain-specific parts in each data center. Two parts disentangle the domain-invariant and domain-specific features from images in each local data center. A server learns a global fPAD model by only aggregating domain-invariant parts of the fPAD models from data centers, and thus, a more generalized fPAD model can be aggregated in server. We introduce the experimental setting to evaluate the proposed FedPAD and FedGPAD frameworks and carry out extensive experiments to provide various insights about federated learning for fPAD.

An Adaptive Spatio-temporal Global Sampling for Presentation Attack Detection

Without developing dedicated countermeasures, facial biometric systems can be spoofed with printed photos, replay attacks, silicone masks, or even a 3D mask of a targeted person. Thus, the threat of presentation attacks needs to be addressed to strengthen the security of the biometric systems. Since a 2D convolutional neural network (CNN) captures static features from video frames, the camera motion might hinders the performance of modern CNNs for video-based presentation attack detection (PAD). Inspired by the egomotion theory, we introduce an adaptive spatiotemporal global sampling (ASGS) technique to compensate the camera motion and use the resulting estimation to encode the appearance and dynamics of the video sequences into a single RGB image. This is achieved by adaptively splitting the video into small segments and capturing their global motion within each segment. The proposed global motion is estimated based on four key steps: dense sampling, FREAK feature extraction and matching, similarity transformation, and aggregation function. This allows using deep models pre-trained on images for video-based PAD detection. Moreover, the interpretation of ASGS reveals that the most important parts for supporting the decision on PAD are consistent with motion cues associated with the artifacts, i.e., hand movement, material reflection, and expression changes. Extensive experiments on four standard face PAD databases demonstrate its effectiveness and encourage further study in this domain.

3sXcsNet: A framework for face presentation attack detection using deep learning

Face Recognition is a widely used authentication method deployed for accessing personal digital devices like mobiles and laptops, digital door locks, or various security applications. However, presentation attacks in form of print, video and mask can circumvent such authentication methods. Several solutions based on texture features are proposed in the state-of-the-art; nevertheless, they are sensitive to varying illumination conditions and are less resilient against 2D and 3D attacks. In this paper, we propose a novel 3-stream compact Xception framework (3sXcsNet) consisting of four input streams RGB, MSRCR, MSRCP, and a depth map to form two specific 3-stream fusion (CRF and CPF) to mitigate the illumination scenarios. These streams provide discriminative features. RGB images have rich texture features; MSRCR and MSRCP images are illumination invariant and preserve better chromaticity applied on a color or intensity channel, respectively. In addition, an auxiliary depth map that provides depth information for each pixel makes it easier to analyze features and recognize whether the input image is from a real person or a spoof medium. The 3-stream CRF and CPF represent the features differently and feed independently to a deep CNN compact Xception with a CBAM network. These feature maps are fused using an average fusion mechanism for classification of real and spoof faces. We utilize both 2D and 3D attack datasets namely, REPLAY-ATTACK, CASIA-FASD, OULU-NPU, and 3DMAD to evaluate the effectiveness of the proposed framework. We attain an EER of 1.27% and 0.19% for CRF fusion, as well as 1.48% and 0.20% for CPF fusion when assessing performance on the CASIA-FASD and REPLAY-ATTACK datasets, respectively. On the OULU-NPU dataset, we obtain the most favorable results for two protocols while employing CRF fusion, resulting in ACER metrics of 0.7% and 3.2±2.7%, respectively. On the other hand, CPF fusion outperformed with an ACER of 1.8±1.0% for one protocol. Our findings on intra-database outperform the current state-of-the-art. We also perform cross-database testing to assess the generalization capability of the framework on 2D and 3D attack datasets. Furthermore, the proposed method outperforms both fusion procedures in terms of error rate HTER for intra-database and cross-database testing on the 3DMAD dataset.

Improving Monocular Facial Presentation-Attack-Detection Robustness with Synthetic Noise Augmentations.

We present a synthetic augmentation approach towards improving monocular face presentation-attack-detection (PAD) robustness to real-world noise additions. Face PAD algorithms secure authentication systems against spoofing attacks, such as pictures, videos, and 2D-inspired masks. Best-in-class PAD methods typically use 3D imagery, but these can be expensive. To reduce application cost, there is a growing field investigating monocular algorithms that detect facial artifacts. These approaches work well in laboratory conditions, but can be sensitive to the imaging environment (e.g., sensor noise, dynamic lighting, etc.). The ideal solution for noise robustness is training under all expected conditions; however, this is time consuming and expensive. Instead, we propose that physics-informed noise-augmentations can pragmatically achieve robustness. Our toolbox contains twelve sensor and lighting effect generators. We demonstrate that our toolbox generates more robust PAD features than popular augmentation methods in noisy test-evaluations. We also observe that the toolbox improves accuracy on clean test data, suggesting that it inherently helps discern spoof artifacts from imaging artifacts. We validate this hypothesis through an ablation study, where we remove liveliness pairs (e.g., live or spoof imagery only for participants) to identify how much real data can be replaced with synthetic augmentations. We demonstrate that using these noise augmentations allows us to achieve better test accuracy while only requiring 30% of participants to be fully imaged under all conditions. These findings indicate that synthetic noise augmentations are a great way to improve PAD, addressing noise robustness while simplifying data collection.

Fairness in face presentation attack detection

Face recognition (FR) algorithms have been proven to exhibit discriminatory behaviors against certain demographic and non-demographic groups, raising ethical and legal concerns regarding their deployment in real-world scenarios. Despite the growing number of fairness studies in FR, the fairness of face presentation attack detection (PAD) has been overlooked, mainly due to the lack of appropriately annotated data. To avoid and mitigate the potential negative impact of such behavior, it is essential to assess the fairness in face PAD and develop fair PAD models. To enable fairness analysis in face PAD, we present a Combined Attribute Annotated PAD Dataset (CAAD-PAD), offering seven human-annotated attribute labels. Then, we comprehensively analyze the fairness of PAD and its relation to the nature of the training data and the Operational Decision Threshold Assignment (ODTA) through a set of face PAD solutions. Additionally, we propose a novel metric, the Accuracy Balanced Fairness (ABF), that jointly represents both the PAD fairness and the absolute PAD performance. The experimental results pointed out that female and faces with occluding features (e.g. eyeglasses, beard, etc.) are relatively less protected than male and non-occlusion groups by all PAD solutions. To alleviate this observed unfairness, we propose a plug-and-play data augmentation method, FairSWAP, to disrupt the identity/semantic information and encourage models to mine the attack clues. The extensive experimental results indicate that FairSWAP leads to better-performing and fairer face PADs in 10 out of 12 investigated cases.

Fully supervised contrastive learning in latent space for face presentation attack detection

Fully supervised contrastive learning in latent space for face presentation attack detection

A new face presentation attack detection method based on face-weighted multi-color multi-level texture features

A new face presentation attack detection method based on face-weighted multi-color multi-level texture features

Face presentation attack detection: Research opportunities and perspectives

The rapid development of biometric methods and their implementation in practice has led to the widespread attacks called spoofing, which are purely biometric vulnerabilities, but are not used in conjunction with other IT security solutions. Although biometric recognition as a branch of computer science dates back to the 1960s, attacks on biometric systems have become more sophisticated since the 2010s due to great advances in pattern recognition. It should be noted that face recognition is the most attractive topic for deceiving recognition systems. Popular presentation attacks, such as print, replay and mask attacks, have demonstrated a high security risk for SOTA face recognition systems. Many Presentation Attack Detection (PAD) methods (also known as face anti-spoofing methods or countermeasures) have been proposed that can automatically detect and mitigate such targeted attacks. The article presents a systematic survey in face anti-spoofing with prognostic trends in this research area. A brief description of 16 outstanding previous surveys on the face PAD field is mentioned, from which it is possible to trace how this scientific topic has developed. SOTA in PAD provides an analysis of a wide range of the PAD methods, which are categorized into two unbalanced groups: digital (feature-based) and physical (sensor-based) methods. Generalization of deep learning methods as a recent trend aimed at improving recognition results requires special attention. This survey presents five types of generalization such as transfer learning, anomaly detection, few-shot and zero-shot learning, auxiliary supervision, and multi-spectral methods. A summary of over than 40 existing 2D/3D face spoofing databases is a guideline for those who want to select databases for experiments. One can also find a description of performance evaluation metrics and testing protocols. In addition, we discuss trends and perspectives in the emerging field of facial biometrics.

Face presentation attack detection performances of facial regions with multi-block LBP features.

Biometric recognition systems are frequently used in daily life although they are vulnerable to attacks. Today, especially the increasing use of face authentication systems has made these systems the target of face presentation attacks (FPA). This has increased the need for sensitive systems detecting the FPAs. Recently surgical masks, frequently used due to the pandemic, directly affect the performance of face recognition systems. Researchers design face recognition systems only from the eye region. This motivated us to evaluate the FPA detection performance of the eye region. Based on this, in cases where the whole face is not visible, the FPA detection performance of other parts of the face has also been examined. Therefore, in this study, FPA detection performances of facial regions of wide face, cropped face, eyes, nose, and mouth was investigated. For this purpose, the facial regions were determined and normalized, and texture features were extracted using powerful texture descriptor local binary patterns (LBP) due to its easy computability and low processing complexity. Multi-block LBP features are used to obtain more detailed texture information. Generally uniform LBP patterns are used for feature extraction in the literature. In this study, the FPA detection performances of both uniform LBP patterns and all LBP patterns were investigated. The size of feature vector is reduced by principal component analysis, and real/fake classification is performed with support vector machines. Experimental results on NUAA, CASIA, REPLAY-ATTACK and OULU-NPU datasets show that the use of all patterns increased the performance of FPA detection.

A survey on face presentation attack detection mechanisms: hitherto and future perspectives.

The advances in human face recognition (FR) systems have recorded sublime success for automatic and secured authentication in diverse domains. Although the traditional methods have been overshadowed by face recognition counterpart during this progress, computer vision gains rapid traction, and the modern accomplishments address problems with real-world complexity. However, security threats in FR-based systems are a growing concern that offers a new-fangled track to the research community. In particular, recent past has witnessed ample instances of spoofing attacks where imposter breaches security of the system with an artifact of human face to circumvent the sensor module. Therefore, presentation attack detection (PAD) capabilities are instilled in the system for discriminating genuine and fake traits and anticipation of their impact on the overall behavior of the FR-based systems. To scrutinize exhaustively the current state-of-the-art efforts, provide insights, and identify potential research directions on face PAD mechanisms, this systematic study presents a review of face anti-spoofing techniques that use computational approaches. The study includes advancements in face PAD mechanisms ranging from traditional hardware-based solutions to up-to-date handcrafted features or deep learning-based approaches. We also present an analytical overview of face artifacts, performance protocols, and benchmark face anti-spoofing datasets. In addition, we perform analysis of the twelve recent state-of-the-art (SOTA) face PAD techniques on a common platform using identical dataset (i.e., REPLAY-ATTACK) and performance protocols (i.e., HTER and ACA). Our overall analysis investigates that despite prevalent face PAD mechanisms demonstrate potential performance, there exist some crucial issues that requisite a futuristic attention. Our analysis put forward a number of open issues such as; limited generalization to unknown attacks, inadequacy of face datasets for DL-models, training models with new fakes, efficient DL-enabled face PAD with smaller datasets, and limited discrimination of handcrafted features. Furthermore, the COVID-19 pandemic is an additional challenge to the existing face-based recognition systems, and hence to the PAD methods. Our motive is to present a complete reference of studies in this field and orient researchers to promising directions.

Asymmetric Modality Translation for Face Presentation Attack Detection

Face presentation attack detection (PAD) is an essentialmeasure to protect face recognition systems from being spoofed by malicious users and has attracted great attention from both academia and industry. Although most of the existing methods can achieve desired performance to some extent, the generalization issue of face presentation attack detection under cross-domain settings (e.g., the setting of unseen attacks and varying illumination) remains to be solved. In this paper, we propose a novel framework based on asymmetric modality translation for face presentation attack detection in bi-modality scenarios. Under the framework, we establish connections between two modality images of genuine faces. Specifically, a novel modality fusion scheme is presented that the image of one modality is translated to the other one through an asymmetric modality translator, then fused with its corresponding paired image. The fusion result is fed as the input to a discriminator for inference. The training of the translator is supervised by an asymmetric modality translation loss. Besides, an illumination normalization module based on Pattern of Local Gravitational Force (PLGF) representation is used to reduce the impact of illumination variation. We conduct extensive experiments on three public datasets, which validate that our method is effective in detecting various types of attacks and achieves state-of-the-art performance under different evaluation protocols.

Unknown Face Presentation Attack Detection via Localized Learning of Multiple Kernels

Unknown Face Presentation Attack Detection via Localized Learning of Multiple Kernels

A Learnable Gradient operator for face presentation attack detection

Face presentation attack detection (PAD) aims to protect the security of face recognition systems. The existing depth-supervised method using stacked vanilla convolutions cannot explicitly extract efficient fine-grained information (e.g., spatial gradient magnitude) for the distinction between bona fide and attack presentations. To address this issue, the Sobel operator has been demonstrated effective to acquire gradient magnitude due to the fast calculation capacity for high-frequency information. However, the Sobel operator is hand-crafted so cannot deal with complex textures. Differently, we develop a learnable gradient operator (LGO) to adaptively learn gradient information in a data-driven way, which is a generalization of existing gradient operators and effectively captures detailed discriminative clues from raw pixels. In parallel, we propose an adaptive gradient loss for better optimization. Extensive experimental comparisons with the state-of-the-art methods on the widely used Replay-Attack, CASIA-FASD, OULU-NPU, and SiW datasets demonstrate the superior performance of the proposed approach.

Fusion Methods for Face Presentation Attack Detection.

Face presentation attacks (PA) are a serious threat to face recognition (FR) applications. These attacks are easy to execute and difficult to detect. An attack can be carried out simply by presenting a video, photo, or mask to the camera. The literature shows that both modern, pre-trained, deep learning-based methods, and traditional hand-crafted, feature-engineered methods have been effective in detecting PAs. However, the question remains as to whether features learned in existing, deep neural networks sufficiently encompass traditional, low-level features in order to achieve optimal performance on PA detection tasks. In this paper, we present a simple feature-fusion method that integrates features extracted by using pre-trained, deep learning models with more traditional colour and texture features. Extensive experiments clearly show the benefit of enriching the feature space to improve detection rates by using three common public datasets, namely CASIA, Replay Attack, and SiW. This work opens future research to improve face presentation attack detection by exploring new characterizing features and fusion strategies.

Deep Transfer Learning on the Aggregated Dataset for Face Presentation Attack Detection

Presentation attacks are becoming a serious threat to one of the most common biometric applications, namely face recognition (FR). In recent years, numerous methods have been presented to detect and identify these attacks using publicly available datasets. However, such datasets are often collected in controlled environments and are focused on one specific type of attack. We hypothesise that a model’s accurate performance on one or more public datasets does not necessarily guarantee generalisation across other, unseen face presentation attacks. To verify our hypothesis, in this paper, we present an experimental framework where the generalisation ability of pre-trained deep models is assessed using four popular and commonly used public datasets. Extensive experiments were carried out using various combinations of these datasets. Results show that, in some circumstances, a slight improvement in model performance can be achieved by combining different datasets for training purposes. However, even with a combination of public datasets, models still could not be trained to generalise to unseen attacks. Moreover, models could not necessarily generalise to a learned format of attack over different datasets. The work and results presented in this paper suggest that more diverse datasets are needed to drive this research as well as the need for devising new methods capable of extracting spoof-specific features which are independent of specific datasets.

Face Presentation Attack Detection Using Deep Background Subtraction

Currently, face recognition technology is the most widely used method for verifying an individual’s identity. Nevertheless, it has increased in popularity, raising concerns about face presentation attacks, in which a photo or video of an authorized person’s face is used to obtain access to services. Based on a combination of background subtraction (BS) and convolutional neural network(s) (CNN), as well as an ensemble of classifiers, we propose an efficient and more robust face presentation attack detection algorithm. This algorithm includes a fully connected (FC) classifier with a majority vote (MV) algorithm, which uses different face presentation attack instruments (e.g., printed photo and replayed video). By including a majority vote to determine whether the input video is genuine or not, the proposed method significantly enhances the performance of the face anti-spoofing (FAS) system. For evaluation, we considered the MSU MFSD, REPLAY-ATTACK, and CASIA-FASD databases. The obtained results are very interesting and are much better than those obtained by state-of-the-art methods. For instance, on the REPLAY-ATTACK database, we were able to attain a half-total error rate (HTER) of 0.62% and an equal error rate (EER) of 0.58%. We attained an EER of 0% on both the CASIA-FASD and the MSU MFSD databases.

Self-supervised 2D face presentation attack detection via temporal sequence sampling

Conventional 2D face biometric systems are vulnerable to presentation attacks performed with different face artefacts, e.g., printouts, video-replays and wearable 3D masks. The research focus in face presentation attack detection (PAD) has been recently shifting towards end-to-end learning of deep representations directly from annotated data rather than designing hand-crafted (low-level) features. However, even the state-of-the-art deep learning based face PAD models have shown unsatisfying generalization performance when facing unknown attacks or acquisition conditions due to lack of representative training and tuning data available in the existing public benchmarks. To alleviate this issue, we propose a video pre-processing technique called Temporal Sequence Sampling (TSS) for 2D face PAD by removing the estimated inter-frame 2D affine motion in the view and encoding the appearance and dynamics of the resulting smoothed video sequence into a single RGB image. Furthermore, we leverage the features of a Convolutional Neural Network (CNN) by introducing a self-supervised representation learning scheme, where the labels are automatically generated by the TSS method as the stabilized frames accumulated over video clips of different temporal lengths provide the supervision. The learnt feature representations are then fine-tuned for the downstream task using labelled face PAD data. Our extensive experiments on four public benchmarks, namely Replay-Attack, MSU-MFSD, CASIA-FASD and OULU-NPU, demonstrate that the proposed framework provides promising generalization capability and encourage further study in this domain.

Face presentation attack detection based on optical flow and texture analysis

Towards the security threats brought by presented fake faces to face recognition systems, many countermeasures have been presented to resist fake faces and achieved promising performance, where facial movement is one of the commonly used cues. However, the more detailed and distinguishable information in the motion cue is not well explored in these methods. In addition, the texture cue used for face presentation attack detection (PAD) is also not well integrated into motion. Therefore, we propose a detection method by analyzing the cues of facial movement and texture. More specifically, the optical flows of a continuous video sequence are first extracted, which can describe the detailed movement direction and movement amplitude. Then, the extracted optical flows are concatenated with the video frames as the input of the network. After that, region and channel attention mechanisms are jointly introduced to adaptively allocate the classification weights. Finally, the fused motion and texture cues are fed into a convolutional network to extract features and identify whether the input video sequence is from live face or not. The proposed detection method is tested on the databases of Replay-Attack, OULU-NPU and HKBU-MARs V1. The experiments show that our proposed face PAD method can well separate various types of fake faces compared to state-of-the-art methods.

Domain-Specific Adaptation of CNN for Detecting Face Presentation Attacks in NIR

For the automotive industry moving towards personalized applications and experiences, the identification of the person inside vehicle is necessary; and it must be carried out in a secure manner. In this paper, we propose a unique face presentation attack detection (PAD) system for operation inside a passenger vehicle. A typical <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-vehicular</i> face PAD system is required to function with several constraints such as bounded sensing (imaging) capabilities, limited computing resources on embedded devices, real-time inference, and essentially, very high accuracy. In this work, we develop a face PAD system for automotive domain, relying on a single NIR camera, to continually verify whether the driver’s face is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bona-fide</i> or not. Our work has two main contributions: first, a lightweight face PAD framework has been developed using a 9-layer convolutional neural network (CNN). With its compact size and limited set of operators, it can be deployed in a resource constrained embedded device to achieve a near real-time inference. To alleviate the problem of limited training data (face PAD in NIR) for a given system, we develop an efficient mechanism to obtain this CNN through the combination of adaptation of domain-specific layers and task-specific fine-tuning of a base CNN. As the second contribution, we collect a large face PAD dataset with 5800+ videos, acquired in NIR (940 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nm</i> ) illumination, for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-vehicular</i> use-cases. This dataset, named VFPAD, captures several real-world variations in terms of environmental settings, illumination, subject’s pose, and appearances. Based on the VFPAD dataset, we demonstrate that the proposed face PAD method achieves very high performance (overall accuracy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx 98.0$ </tex-math></inline-formula> %), and also outperforms several baseline face PAD methods. The dataset will be shared with the wider scientific community for research purposes.