Face Anti-spoofing Research Articles

M3FAS: An Accurate and Robust MultiModal Mobile Face Anti-Spoofing System

M3FAS: An Accurate and Robust MultiModal Mobile Face Anti-Spoofing System

A Domain Generalized Face Anti-Spoofing System Using Domain Adversarial Learning

This research addresses the enhancement of face anti-spoofing (FAS) in facial recognition systems (FRS) against sophisticated fraudulent activities. Prior methods primarily focus on extracting facial features like color, texture, and dynamic variations, yet these methods struggle to accurately identify common characteristics of forged faces, thereby limiting generalization in practical scenarios. This study aims to propose a novel representation learning framework incorporating adversarial learning algorithms, to segregate features into liveness-specific and domain-specific categories, emphasizing liveness-specific features for FAS advancement. Feature disentanglement is central to this approach, enabling the deep learning models to effectively discern separable latent generating factors, such as identity, liveness, appearance, and texture. This methodology enhances model interpretability, explainability, and generalization. Additionally, Grad-CAM is employed to elucidate the basis for classifications made by the architecture, increasing explainability and trustworthiness. Empirical evaluation across panoply available FAS datasets confirms the superiority, significantly improving performance and robustness over existing technologies.

Efficient face anti-spoofing via head-aware transformer based knowledge distillation with 5 MB model parameters

Although face recognition technology has been applied in many scenarios, it still suffers from many types of presentation attacks, so face anti-spoofing (FAS) becomes a hot topic in computer vision. Recently, vision transformer is recognized as the mainstream architecture for FAS, which always relies on auxiliary information, sophisticated tricks and huge model parameters. Considering that face based identity authentication usually takes place on mobile-like devices, therefore how to design an effective and lightweight model is of great significance. Inspired by the powerful global modeling ability of self-attention and the model compression ability of knowledge distillation, a simple yet effective knowledge distillation approach is proposed for FAS under transformer framework. Our primary idea is to leverage the rich knowledge of a teacher network pre-trained on large-scale face data to guide the learning of a lightweight student network. The main contributions of our method are threefold: (1) Feature- and logits-level distillation are combined to transfer the rich knowledge of teacher to student. (2) A head-aware strategy is proposed to deal with the dimension mismatching issue of middle encoder layers between teacher and student networks, in which a novel attention head correlation matrix is introduced. (3) Our method can bridge the performance gap between teacher and student, and the resulting student network is extremely lightweight with only 5 MB parameters. Extensive experiments are conducted on three public face-spoofing datasets, CASIA-FASD, Replay-Attack and OULU-NPU, the results demonstrate that our method can obtain performance on par with or superior to most FAS methods and outperform many knowledge distillation methods. Meanwhile, the distilled student network achieves excellent performance with 17× fewer parameters and 9× faster inference time compared to the teacher network. The code will be publicly available at https://github.com/Maricle-zhangjun/HaTFAS.

Source-Free Domain Adaptation With Domain Generalized Pretraining for Face Anti-Spoofing.

Source-free domain adaptation (SFDA) shows the potential to improve the generalizability of deep learning-based face anti-spoofing (FAS) while preserving the privacy and security of sensitive human faces. However, existing SFDA methods are significantly degraded without accessing source data due to the inability to mitigate domain and identity bias in FAS. In this paper, we propose a novel Source-free Domain Adaptation framework for FAS (SDA-FAS) that systematically addresses the challenges of source model pre-training, source knowledge adaptation, and target data exploration under the source-free setting. Specifically, we develop a generalized method for source model pre-training that leverages a causality-inspired PatchMix data augmentation to diminish domain bias and designs the patch-wise contrastive loss to alleviate identity bias. For source knowledge adaptation, we propose a contrastive domain alignment module to align conditional distribution across domains with a theoretical equivalence to adaptation based on source data. Furthermore, target data exploration is achieved via self-supervised learning with patch shuffle augmentation to identify unseen attack types, which is ignored in existing SFDA methods. To our best knowledge, this paper provides the first full-stack privacy-preserving framework to address the generalization problem in FAS. Extensive experiments on nineteen cross-dataset scenarios show our framework considerably outperforms state-of-the-art methods.

A MobileFaceNet-Based Face Anti-Spoofing Algorithm for Low-Quality Images

The Face Anti-Spoofing (FAS) methods plays a very important role in ensuring the security of face recognition systems. The existing FAS methods perform well in short-distance scenarios, e.g., phone unlocking, face payment, etc. However, it is still challenging to improve the generalization of FAS in long-distance scenarios (e.g., surveillance) due to the varying image quality. In order to address the lack of low-quality images in real scenarios, we build a Low-Quality Face Anti-Spoofing Dataset (LQFA-D) by using Hikvision’s surveillance cameras. In order to deploy the model on an edge device with limited computation, we propose a lightweight FAS network based on MobileFaceNet, in which the Coordinate Attention (CA) attention model is introduced to capture the important spatial information. Then, we propose a multi-scale FAS framework for low-quality images to explore multi-scale features, which includes three multi-scale models. The experimental results of the LQFA-D show that the Average Classification Error Rate (ACER) and detection time of the proposed method are 1.39% and 45 ms per image for the low-quality images, respectively. It demonstrates the effectiveness of the proposed method in this paper.

CG-FAS: Cross-label Generative Augmentation for Face Anti-Spoofing

CG-FAS: Cross-label Generative Augmentation for Face Anti-Spoofing

Rethinking Vision Transformer and Masked Autoencoder in Multimodal Face Anti-Spoofing

Recently, vision transformer (ViT) based multimodal learning methods have been proposed to improve the robustness of face anti-spoofing (FAS) systems. However, there are still no works to explore the fundamental natures (e.g., modality-aware inputs, suitable multimodal pre-training, and efficient finetuning) in vanilla ViT for multimodal FAS. In this paper, we investigate three key factors (i.e., inputs, pre-training, and finetuning) in ViT for multimodal FAS with RGB, Infrared (IR), and Depth. First, in terms of the ViT inputs, we find that leveraging local feature descriptors (such as histograms of oriented gradients) benefits the ViT on IR modality but not RGB or Depth modalities. Second, in consideration of the task (FAS vs. generic object classification) and modality (multimodal vs. unimodal) gaps, ImageNet pre-trained models might be sub-optimal for the multimodal FAS task. Finally, in observation of the inefficiency on direct finetuning the whole or partial ViT, we design an adaptive multimodal adapter (AMA), which can efficiently aggregate local multimodal features while freezing majority of ViT parameters. To bridge these gaps, we propose the modality-asymmetric masked autoencoder (M2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{2}$$\\end{document}A2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{2}$$\\end{document}E) for multimodal FAS self-supervised pre-training without costly annotated labels. Compared with the previous modality-symmetric autoencoder, the proposed M2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{2}$$\\end{document}A2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{2}$$\\end{document}E is able to learn more intrinsic task-aware representation and compatible with modality-agnostic (e.g., unimodal, bimodal, and trimodal) downstream settings. Extensive experiments with both unimodal (RGB, Depth, IR) and multimodal (RGB+Depth, RGB+IR, Depth+IR, RGB+Depth+IR) settings conducted on multimodal FAS benchmarks demonstrate the superior performance of the proposed methods. One highlight is that the proposed method is robust under various missing-modality cases where previous multimodal FAS models suffer serious performance drops. We hope these findings and solutions can facilitate the future research for ViT-based multimodal FAS.

CSDG-FAS: Closed-Space Domain Generalization for Face Anti-spoofing

CSDG-FAS: Closed-Space Domain Generalization for Face Anti-spoofing

Anomaly Metrics on Class Variations For Face Anti-Spoofing

Abstract In face anti-spoofing tasks, distinguishing between live and spoof faces across different data domains presents challenges due to inter-class similarities, intra-class variations and unknown spoof patterns. This hampers generalization in real-world applications. To address this, we propose a novel convolutional neural network framework that utilizes spatial-frequency cues for 2D and 3D attacks. Furthermore, we introduce compact anomaly metrics and design three anomaly metrics-based supervisions from the perspective of Reed-Xiaoli anomaly detection, aiming to tackle the challenge posed by unknown attacks. Thanks to our proposed spatial frequency factorization network and its frequency-related supervisions, the spoofing cues are significantly enhanced, resulting in remarkable improvements in our experimental results. These outcomes demonstrate that our proposed framework achieves state-of-the-art performance on both monocular and multi-spectral benchmark datasets.

Face anti-spoofing with cross-stage relation enhancement and spoof material perception

Face Anti-Spoofing (FAS) seeks to protect face recognition systems from spoofing attacks, which is applied extensively in scenarios such as access control, electronic payment, and security surveillance systems. Face anti-spoofing requires the integration of local details and global semantic information. Existing CNN-based methods rely on small stride or image patch-based feature extraction structures, which struggle to capture spatial and cross-layer feature correlations effectively. Meanwhile, Transformer-based methods have limitations in extracting discriminative detailed features. To address the aforementioned issues, we introduce a multi-stage CNN-Transformer-based framework, which extracts local features through the convolutional layer and long-distance feature relationships via self-attention. Based on this, we proposed a cross-attention multi-stage feature fusion, employing semantically high-stage features to query task-relevant features in low-stage features for further cross-stage feature fusion. To enhance the discrimination of local features for subtle differences, we design pixel-wise material classification supervision and add a auxiliary branch in the intermediate layers of the model. Moreover, to address the limitations of a single acquisition environment and scarcity of acquisition devices in the existing Near-Infrared dataset, we create a large-scale Near-Infrared Face Anti-Spoofing dataset with 380k pictures of 1040 identities. The proposed method could achieve the state-of-the-art in OULU-NPU and our proposed Near-Infrared dataset at just 1.3GFlops and 3.2M parameter numbers, which demonstrate the effective of the proposed method.

Domain-Hallucinated Updating for Multi-Domain Face Anti-spoofing

Multi-Domain Face Anti-Spoofing (MD-FAS) is a practical setting that aims to update models on new domains using only novel data while ensuring that the knowledge acquired from previous domains is not forgotten. Prior methods utilize the responses from models to represent the previous domain knowledge or map the different domains into separated feature spaces to prevent forgetting. However, due to domain gaps, the responses of new data are not as accurate as those of previous data. Also, without the supervision of previous data, separated feature spaces might be destroyed by new domains while updating, leading to catastrophic forgetting. Inspired by the challenges posed by the lack of previous data, we solve this issue from a new standpoint that generates hallucinated previous data for updating FAS model. To this end, we propose a novel Domain-Hallucinated Updating (DHU) framework to facilitate the hallucination of data. Specifically, Domain Information Explorer learns representative domain information of the previous domains. Then, Domain Information Hallucination module transfers the new domain data to pseudo-previous domain ones. Moreover, Hallucinated Features Joint Learning module is proposed to asymmetrically align the new and pseudo-previous data for real samples via dual levels to learn more generalized features, promoting the results on all domains. Our experimental results and visualizations demonstrate that the proposed method outperforms state-of-the-art competitors in terms of effectiveness.

MF 2 ShrT: Multimodal Feature Fusion Using Shared Layered Transformer for Face Anti-spoofing

In recent times, Face Anti-spoofing (FAS) has gained significant attention in both academic and industrial domains. Although various convolutional neural network (CNN)-based solutions have emerged, multimodal approaches incorporating RGB, depth, and information retrieval (IR) have exhibited better performance than unimodal classifiers. The increasing veracity of modern presentation attack instruments results in a persistent need to enhance the performance of such models. Recently, self-attention-based vision transformers (ViT) have become a popular choice in this field. Their fundamental aspects for multimodal FAS have not been thoroughly explored yet. Therefore, we propose a novel framework for FAS called MF 2 ShrT, which is based on a pretrained vision transformer. The proposed framework uses overlap patches and parameter sharing in the ViT network, allowing it to utilize multiple modalities in a computationally efficient manner. Furthermore, to effectively fuse intermediate features from different encoders of each ViT, we explore a T-encoder-based hybrid feature block enabling the system to identify correlations and dependencies across different modalities. MF 2 ShrT outperforms conventional vision transformers and achieves state-of-the-art performance on benchmarks CASIA-SURF and WMCA, demonstrating the efficiency of transformer-based models for presentation attack detection PAD).

Retracted: DTFA-Net: Dynamic and Texture Features Fusion Attention Network for Face Antispoofing

Retracted: DTFA-Net: Dynamic and Texture Features Fusion Attention Network for Face Antispoofing

Fourier-Based Frequency Space Disentanglement and Augmentation for Generalizable Face Anti-Spoofing.

Generalizing face anti-spoofing (FAS) models to unseen distributions is challenging due to domain shifts. Previous domain generalization (DG) based FAS methods focus on learning invariant features across domains in the spatial space, which may be ineffective in detecting subtle spoof patterns. In this paper, we propose a novel approach called Frequency Space Disentanglement and Augmentation (FSDA) for generalizable FAS. Specifically, we leverage Fourier transformation to analyze face images in the frequency space, where the amplitude spectrum captures low-level texture information that forms distinct visual appearances, and the phase spectrum corresponds to the content information. We hypothesize that the liveness of a face is more related to these low-level patterns rather than high-level content information. To locate spoof traces, we disentangle the amplitude spectrum into domain-related and spoof-related components using either empirical or learnable strategies. We then propose a frequency space augmentation technique that mixes the disentangled components of two images to synthesize new variations. By imposing a distillation loss and a consistency loss on the augmented samples, our model learns to capture spoof patterns that are robust to both domain and spoof type variations. Extensive experiments on four FAS datasets demonstrate the superiority of our method in improving the generalization ability of FAS models in various unseen scenarios.

Learning deep forest for face anti-spoofing: An alternative to the neural network against adversarial attacks

<p>Face anti-spoofing (FAS) is significant for the security of face recognition systems. neural networks (NNs), including convolutional neural network (CNN) and vision transformer (ViT), have been dominating the field of the FAS. However, NN-based methods are vulnerable to adversarial attacks. Attackers could insert adversarial noise into spoofing examples to circumvent an NN-based face-liveness detector. Our experiments show that the CNN or ViT models could have at least an 8% equal error rate (EER) increment when encountering adversarial examples. Thus, developing methods other than NNs is worth exploring to improve security at the system level. In this paper, we have proposed a novel solution for FAS against adversarial attacks, leveraging a deep forest model. Our approach introduces a multi-scale texture representation based on local binary patterns (LBP) as the model input, replacing the grained-scanning mechanism (GSM) used in the traditional deep forest model. Unlike GSM, which scans raw pixels and lacks discriminative power, our LBP-based scheme is specifically designed to capture texture features relevant to spoofing detection. Additionally, transforming the input from the RGB space to the LBP space enhances robustness against adversarial noise. Our method achieved competitive results. When testing with adversarial examples, the increment of EER was less than 3%, more robust than CNN and ViT. On the benchmark database IDIAP REPLAY-ATTACK, a 0% EER was achieved. This work provides a competitive option in a fusing scheme for improving system-level security and offers important ideas to those who want to explore methods besides CNNs. To the best of our knowledge, this is the first attempt at exploiting the deep forest model in the problem of FAS, with the consideration of adversarial attacks.</p>

Domain-Adaptive Energy-Based Models for Generalizable Face Anti-Spoofing

Domain-Adaptive Energy-Based Models for Generalizable Face Anti-Spoofing

Category-Conditional Gradient Alignment for Domain Adaptive Face Anti-Spoofing

Category-Conditional Gradient Alignment for Domain Adaptive Face Anti-Spoofing

Dual Teacher Knowledge Distillation with Domain Alignment for Face Anti-spoofing

Dual Teacher Knowledge Distillation with Domain Alignment for Face Anti-spoofing

MFAE: Masked Frequency Autoencoders for Domain Generalization Face Anti-Spoofing

MFAE: Masked Frequency Autoencoders for Domain Generalization Face Anti-Spoofing

Data augmentation based face anti-spoofing (FAS) scheme using deep learning techniques

Recent years have seen a rise in interest in face anti-spoofing (FAS) owing to the critical function it plays in protecting face recognition systems against presentation assaults (PAs). Early-stage FAS approaches relying on handmade characteristics become inaccurate when steadily realistic PAs of unique sorts emerge. Thus, face anti-spoofing algorithms are gaining increasing relevance in such setups. A very innovative method called deep learning has shown remarkable success in difficult computer vision problems. The proposed method uses deep acquisition and transfer of learning to extract characteristics from people’s faces. This is why the authors of this study recommend using the Faster RCNN classifier with a face-liveness detection approach. Two distinct components— the data augmentation module for assessing sparse information as well as the faster RCNN classifier module— make up the anti-spoofing approach. We may use any publicly accessible dataset to train our quicker RCNN classifier. We successively fused these two components and used the Android platform to create a basic face recognition app. The results of the tests demonstrate that the developed module can identify several types of face spoof assaults, such as those carried out with the use of posters, masks, or cell phones. Testing the proposed architecture both across and inside databases using three benchmarking (Idiap Replay Attack, CASIA- FASD, & 3DMAD) demonstrate its ability to deliver outcomes on par with cutting-edge techniques.