Face Spoofing Research Articles

A systematic review on face spoofing detection methods

Face biometric researchers have had a lot of success over the past 62 years. Face recognition technology has been widely employed in both commercial and governmental applications, including mobile, banking, and surveillance systems. Though, the facial recognition system's ability to survive an uninvited attacker is a crucial issue. Face recognition software is susceptible to fake images and video attacks. Anti-spoofing solutions are useful in these situations for thwarting these attacks. The biometric community as a whole, including researchers, developers, and retailers, has spent the last ten years working on difficult projects to create a more precise defence against spoofing attacks. Despite numerous face anti-spoofing or liveness detection techniques being put out, the problem has not been addressed since it is challenging to identify the features and techniques for spoof assaults. This paper's objective is to present a thorough analysis of antispoofing techniques. The study came to the conclusion that in order to increase the system's security, computational efficiency, and dependability, it is necessary to provide more generalized methods for the detection of unanticipated spoofing assaults.

A Color-Texture-Based Deep Neural Network Technique to Detect Face Spoofing Attacks

Abstract Given the face spoofing attack, adequate protection of human identity through face has become a significant challenge globally. Face spoofing is an act of presenting a recaptured frame before the verification device to gain illegal access on behalf of a legitimate person with or without their concern. Several methods have been proposed to detect face spoofing attacks over the last decade. However, these methods only consider the luminance information, reflecting poor discrimination of spoofed face from the genuine face. This article proposes a practical approach combining Local Binary Patterns (LBP) and convolutional neural network-based transfer learning models to extract low-level and high-level features. This paper analyzes three color spaces (i.e., RGB, HSV, and YCrCb) to understand the impact of the color distribution on real and spoofed faces for the NUAA benchmark dataset. In-depth analysis of experimental results and comparison with other existing approaches show the superiority and effectiveness of our proposed models.

A new hybrid CNN–LSTM model with Non-Softmax functions for face spoof detection

With the rapid technological development of video transmission, it is applied on various applications such as security, forgery detection and surveillance. Moreover, the images and videos supply a huge volume of variations in intra-personal and also make the face recognition significant in a biometric security area. In addition, automatic face recognition process is a widely applied concept in security. The spoofing attacks account for reproducing a human face by applying photographs or videos. The face recognition and spoof detection processes are performed by using machine learning and deep learning algorithms by analysing the images in videos. For the purpose of enhancing the prediction accuracy, we propose a new hybrid deep learning technique called hybrid convolutional neural network (CNN)-based architecture with long short-term memory (LSTM) units to study the impact in classification. This hybrid approach uses a non-softmax function for making effective decision on classification. The experiments have been performed for evaluating the proposed technique and proved better than the existing deep learning algorithms in terms of precision, recall, f-measure and accuracy.

Face Spoofing, Age, Gender and Facial Expression Recognition Using Advance Neural Network Architecture-Based Biometric System.

Nowadays, the demand for soft-biometric-based devices is increasing rapidly because of the huge use of electronics items such as mobiles, laptops and electronic gadgets in daily life. Recently, the healthcare department also emerged with soft-biometric technology, i.e., face biometrics, because the entire data, i.e., (gender, age, face expression and spoofing) of patients, doctors and other staff in hospitals is managed and forwarded through digital systems to reduce paperwork. This concept makes the relation friendlier between the patient and doctors and makes access to medical reports and treatments easier, anywhere and at any moment of life. In this paper, we proposed a new soft-biometric-based methodology for a secure biometric system because medical information plays an essential role in our life. In the proposed model, 5-layer U-Net-based architecture is used for face detection and Alex-Net-based architecture is used for classification of facial information i.e., age, gender, facial expression and face spoofing, etc. The proposed model outperforms the other state of art methodologies. The proposed methodology is evaluated and verified on six benchmark datasets i.e., NUAA Photograph Imposter Database, CASIA, Adience, The Images of Groups Dataset (IOG), The Extended Cohn-Kanade Dataset CK+ and The Japanese Female Facial Expression (JAFFE) Dataset. The proposed model achieved an accuracy of 94.17% for spoofing, 83.26% for age, 95.31% for gender and 96.9% for facial expression. Overall, the modification made in the proposed model has given better results and it will go a long way in the future to support soft-biometric based applications.

Face Anti-Spoofing Using Transformers With Relation-Aware Mechanism

Face anti-spoofing (FAS) is important to secure face recognition systems. Deep learning has obtained great success in this area, however, most existing approaches fail to consider comprehensive relation-aware local representations of live and spoof faces. To address this issue, we propose a Transformer-based Face Anti-Spoofing (TransFAS) model to explore comprehensive facial parts for FAS. Besides the multi-head self-attention which explores relations among local patches in the same layer, we propose cross-layer relation-aware attentions (CRA) to adaptively integrate local patches from different layers. Furthermore, to effectively fuse hierarchical features, we explore the best hierarchical feature fusion (HFF) structure, which can capture the complementary information between low-level artifacts and high-level semantic features for the spoofing patterns. With these novel modules, TransFAS not only improves the generalization capability of the classical vision transformer, but also achieves SOTA performance on multiple benchmarks, demonstrating the superiority of the transformer-based model for FAS.

Federated Multitask Learning with Manifold Regularization for Face Spoof Attack Detection ∗

Face recognition has been widely used in personal authentication, especially on edge computing devices. However, face recognition systems suffer from face spoof attack. In this paper, a novel method for face spoof attack detection in edge computing scenarios is proposed. It is based on federated learning and improves traditional federated learning with multitask learning and manifold regularization, which is known as federated learning for face spoof attack detection (FedFSAD). In this way, local model learning is completed on edge devices and global model learning only depends on the trained local models without using the original image data. Besides, the performance is improved by imposing hypergraph manifold regularization in the global training of multitask learning. The results of comprehensive experiments show that the detection performance is improved by about 10% and robust against stragglers and network delays, which indicates the effectiveness of FedFSAD.

Face spoofing detection ensemble via multistage optimisation and pruning

Despite the recent improvements in facial recognition, face spoofing attacks can still pose a serious security threat to biometric systems. As fraudsters are coming up with novel spoofing attacks, anomaly-based detectors, compared to the binary spoofing attack counterparts, have certain generalisation performance advantages. In this work, we investigate the merits of fusing multiple anomaly classifiers using weighted averaging (WA) fusion. The design of the entire system is based on genuine-access data only. To optimise the parameters of WA, we propose a novel three-stage optimisation method with the following contributions: (a) A new hybrid optimisation method using Genetic Algorithm (GA) and Pattern Search (PS) to explore the weight space more effectively (b) a novel two-sided score normalisation method to improve the anomaly detection performance (c) a new ensemble pruning method to improve the generalisation performance. To further boost the performance of the proposed anomaly detection ensemble, we incorporate client-specific information to train the proposed model. We evaluate the capability of the proposed model on publicly available face spoofing databases including Replay-Attack, Replay-Mobile and Rose-Youtu. The experimental results demonstrate that the proposed WA fusion outperforms the state-of-the-art anomaly-based and multiclass approaches.

A remote PPG based approach for heart rate estimation

Heart rate (HR) estimation is an essential physiological process in the field of biosignal analysis. Recently, remote Photoplethysmography (rPPG) is a pathbreaking development in this field wherein the PPG signal is extracted from non-contact face videos. This study, aiming at building a low cost rPPG system for both heart rate estimation and detecting real face, proposes a non-contact, automatic technique of heart rate measurement from video based on the signal processing techniques. The heart rate is remotely measured from a video recording of a person's face using the camera from a smartphone. At the same time, the heart rate extracted from the videos recorded by a smartphone camera is compared with the data obtained from the PPG optical absorption sensor and achieved a similar result. The system can estimate the heart rates from faces from single subject or multiple participants, and can be applied for anti spoofing face recognition.

Effect of Facial Shape Information Reflected on Learned Features in Face Spoofing Detection

<p>Face recognition is a convenient and non-contact biometric method used widely for secure personal authentication. However, the face is an exposed body part, and face spoofing attacks, which compromise the security of systems that use face recognition for authentication, are frequently reported. Previous face spoofing attack detection studies proposed texture-analysis-based methods using handcrafted features or learned features to prevent spoofing attacks. However, it is unclear whether spoofing attack images reflect the face distortion resulting from failing to reflect the three-dimensional structure of a real face. To resolve this problem, we compared and analyzed the face spoofing attack detection performances of two typical convolutional neural network models, namely ResNet-18 and DenseNet-121. CASIA-FASD, Replay-Attack, and PR-FSAD were used as the training data. The classification performance of the model was evaluated based on four protocols. DenseNet-121 exhibited better performance in most scenarios. DenseNet-121 reflected facial shape information well by uniformly applying the learned features of both the initial and final layers during training. It is expected that this study will support the realization of spoofing technology with enhanced security.</p> <p> </p>

Deteksi Non-Spoofing Wajah pada Video secara Real Time Menggunakan Faster R-CNN

Face non-spoofing detection is an important job used to ensure authentication security by performing an analysis of the captured faces. Face spoofing is the process of fake faces by other people to gain illegal access to the biometric system which can be done by displaying videos or images of someone's face on the monitor screen or using printed images. There are various forms of attacks that can be carried out on the face authentication system in the form of face sketches, face photos, face videos and 3D face masks. Such attacks can occur because photos and videos of faces from users of the facial authentication system are very easy to obtain via the internet or cameras. To solve this problem, in this research proposes a non-spoofing face detection model on video using Faster R-CNN. The results obtained in this study are the Faster R-CNN model that can detect non-spoof and spoof face in real time using the Raspberry Pi as a camera with a frame rate of 1 fps.

Face Biometric Spoof Detection Method Using a Remote Photoplethysmography Signal.

Spoofing attacks in face recognition systems are easy because faces are always exposed. Various remote photoplethysmography-based methods to detect face spoofing have been developed. However, they are vulnerable to replay attacks. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that minimizes the susceptibility to certain database dependencies and high-quality replay attacks without additional devices. The proposed method has the following advantages. First, because only an RGB camera is used to detect spoofing attacks, the proposed method is highly usable in various mobile environments. Second, solutions are incorporated in the method to obviate new attack scenarios that have not been previously dealt with. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that improves susceptibility to certain database dependencies and high-quality replay attack, which are the limitations of previous methods without additional devices. In the experiment, we also verified the cut-off attack scenario in the jaw and cheek area where the proposed method can be counter-attacked. By using the time series feature and the frequency feature of the remote photoplethysmography signal, it was confirmed that the accuracy of spoof detection was 99.7424%.

Hybrid Algorithm for Face Spoof Detection

Abstract: The face spoof detection is the approach which can detect spoofed face. The face spoof detection methods has various phases which include pre-processing, feature extraction and classification. The classification algorithm can classify into two classes which are spoofed or not spoofed. The KNN approach is used previously with the GLCM algorithm for the face spoof detection which give low accuracy. In this research work, the hybrid classification method is proposed which is the combination of random forest, k nearest neighbour and SVM Classifiers. The simulation outcomes depict that the introduced method performs more efficiently in comparison with the conventional techniques with regard to accuracy. Keywords: Face Spoof, KNN, Hybrid Classifier, GLCM

Dynamic face authentication systems: Deep learning verification for camera close-Up and head rotation paradigms

The popularity of face-authentication systems has also generated interest in the study of malicious authentication attempts, such as face spoofing attacks. In this study we investigate two dynamic face-authentication challenges: the camera close-up, and head-rotation paradigms. For each paradigm we developed an ML-based face-authentication system that performs the tasks of liveness detection and face verification. In order to generate structured data-representations from the videos collected in the wild, we designed feature representations that extract three-dimensional and spatial characteristics of a face, while also capturing the particular liveness cues of the requested challenge-based movements. Furthermore, a set of Neural-Network models that employ Convolutional Neural Networks and Siamese Neural Network architectures were proposed. To train and test our models we collected a dataset of 177 live videos recorded by 41 different subjects and a set of 243 attack attempts in uncontrolled scenarios. The resulting NN models yield good performance against multiple types of media-based attacks (printed-attacks, screen-attacks, 2D-masks, videos acquired from public social media, deep fakes). The camera close-up system presented an overall liveness detection accuracy of 97.7% and a face verification accuracy of 97.6%. On the other hand, the evaluation of the head-rotation system resulted in a liveness detection accuracy of 92.4% and face verification accuracy of 98.1%. Face authentication methods based on dynamic user-challenges constitute a scalable approach that does not require specialized hardware to increase the security of face authentication systems in realistic usage scenarios. The proposed methods not only make it harder for attackers to generate spoofing attacks, but they also constitute a practical complement to static biometric authentication systems.

Spoof Trace Disentanglement for Generic Face Anti-Spoofing.

Prior studies show that the key to face anti-spoofing lies in the subtle image patterns, termed "spoof trace," e.g., color distortion, 3D mask edge, and Moiré pattern. Spoof detection rooted on those spoof traces can improve not only the model's generalization but also the interpretability. Yet, it is a challenging task due to the diversity of spoof attacks and the lack of ground truth for spoof traces. In this work, we propose a novel adversarial learning framework to explicitly estimate the spoof related patterns for face anti-spoofing. Inspired by the physical process, spoof faces are disentangled into spoof traces and the live counterparts in two steps: additive step and inpainting step. This two-step modeling can effectively narrow down the searching space for adversarial learning of spoof trace. Based on the trace modeling, the disentangled spoof traces can be utilized to reversely construct new spoof faces, which is used as data augmentation to effectively tackle long-tail spoof types. In addition, we apply frequency-based image decomposition in both the input and disentangled traces to better reflect the low-level vision cues. Our approach demonstrates superior spoof detection performance on 3 testing scenarios: known attacks, unknown attacks, and open-set attacks. Meanwhile, it provides a visually-convincing estimation of the spoof traces. Source code and pre-trained models will be publicly available upon publication.

Beyond the Pixel World: A Novel Acoustic-Based Face Anti-Spoofing System for Smartphones

2D face presentation attacks are one of the most notorious and pervasive face spoofing types, which have caused pressing security issues to facial authentication systems. While RGB-based face anti-spoofing (FAS) models have proven to counter the face spoofing attack effectively, most existing FAS models suffer from the overfitting problem (i.e., lack generalization capability to data collected from an unseen environment). Recently, many models have been devoted to capturing auxiliary information (<i>e.g</i>., depth and infrared maps) to achieve a more robust face liveness detection performance. However, these methods require expensive sensors and cost extra hardware to capture the specific modality information, limiting their applications in practical scenarios. To tackle these problems, we devise a novel and cost-effective FAS system based on the acoustic modality, named Echo-FAS, which employs the crafted acoustic signal as the probe to perform face liveness detection. We first propose to build a large-scale, high-diversity, and acoustic-based FAS database, Echo-Spoof. Then, based upon Echo-Spoof, we propose designing a novel two-branch framework that combines the global and local frequency clues of input signals to distinguish inputs, live vs. spoofing faces accurately. The devised Echo-FAS comprises the following three merits: (1) It only needs one available speaker and microphone as sensors while not requiring any expensive hardware; (2) It can successfully capture the 3D geometrical information of input queries and achieve a remarkable face anti-spoofing performance; and (3) It can be handily allied with other RGB-based FAS models to mitigate the overfitting problem in the RGB modality and make the FAS model more accurate and robust. Our proposed Echo-FAS provides new insights regarding the development of FAS systems for mobile devices.

Learning Multi-Granularity Temporal Characteristics for Face Anti-Spoofing

Face anti-spoofing (FAS) is essential for securing face recognition systems. Despite the decent performance, few existing works fully leverage temporal information. This would inevitably lead to inferior performance because real and fake faces tend to share highly similar spatial appearances, while important temporal features between consecutive frames are neglected. In this work, we propose a temporal transformer network (TTN) to learn multi-granularity temporal characteristics for FAS. It mainly consists of temporal difference attentions (TDA), a pyramid temporal aggregation (PTA), and a temporal depth difference loss (TDL). Firstly, the vision transformer (ViT) is used as the backbone where comprehensive local patches are utilized to provide subtle differences between live and spoof faces. Then, instead of learning temporal features on global faces which may miss some important local cues, the TDA is developed to extract motion-sensitive cues on each of the comprehensive local patches. Moreover, the TDA is inserted into different layers of the ViT, learning multi-scale motion-sensitive local cues to improve the FAS performance. Secondly, it is observed that different subjects may have different visual tempos in some actions, making it necessary to model different temporal speeds. Our PTA aggregates temporal features at various tempos, which could build short-range and long-range relations among multiple frames. Thirdly, depth maps for real parts may change continuously, while they remain zeros for spoof regions. In order to locate motion features on facial parts, the TDL is proposed to guide the network to locate spoof facial parts where motion patterns between neighboring frames are set as the ground truth. To the best of our knowledge, this work is the first attempt to learn temporal characteristics via transformers. Both qualitative and quantitative results on several challenging tasks demonstrate the usefulness and effectiveness of our proposed methods.

An Investigation on the Effect of Color Channels on Cnn Architectures for Face Spoofing Detection

An Investigation on the Effect of Color Channels on Cnn Architectures for Face Spoofing Detection

SUNRISE: Improving 3D mask Face Anti-spoofing for Short Videos using Pre-emptive Split and Merge

In the current digital world, face analytic systems become an integral part of daily life. However, todays cutting-edge technology and readily available social media information make these systems vulnerable through face spoofing attacks. These attacks can be mitigated using remote Photoplethysmography (rPPG), which remotely detects cardiovascular signals. Unfortunately, the illumination variation and face deformations can easily corrupt the pulse signals and thereby degrade the performance of rPPG-based anti-spoofing methods, even when longer-duration face videos are employed. This paper proposes a face anti-spoofing method <b>SUNRISE</b>, that is, <b>S</b>hort videos <b>U</b>si<b>N</b>g p<b>R</b>e-empt<b>I</b>ve <b>S</b>plit and m<b>E</b>rge. It is an rPPG-based face anti-spoofing for short duration videos wherein facial deformations are removed by introducing the split and merge mechanism. It splits the video into several clips, provides low importance to the clips containing facial deformations, and eventually merges the results using quality-based fusion. The efficacy of existing rPPG-based methods is limited because they employed high dimensional features for training using the limited training data. We mitigate this limitation by utilizing the statistical features of clips. The experimental results on publicly available datasets reveal that the proposed method exhibits significantly better performance than the well-known existing methods for a short time duration

Face Anti-Spoofing System using Motion and Similarity Feature Elimination under Spoof Attacks

From border control to mobile device unlocking applications, the practical utility of biometric system can be seriously compromised due face spoofing attacks. So, face recognition systems require greater attention to combating face spoofing attacks. As, face spoofing attacks can be easily propelled through 3D masks, video replays, and printed photos so we are presented face recognition system using motion and similarity features elimination under spoof attacks against the Replay Attack and Institute of Automation, Chinese Academy of Sciences (CASIA) databases. In this paper a calculative analysis has been done by firstly segmenting the foreground and background regions from the input video using Gaussion Mixture Model and secondly by extracting features i.e., face, eye, and nose and applied 26 image quality assessment parameters on spoof face videos under different illumination lighting conditions. The results attained using Replay Attack and CASIA databases are extremely competitive in discriminating from fake traits with paralleled viz-a-viz other approaches. Different machine learning classifiers and their comparative analysis with existing approaches has been shown.

Developing a generic framework for anomaly detection

The fusion of one-class classifiers (OCCs) has been shown to exhibit promising performance in a variety of machine learning applications. The ability to assess the similarity or correlation between the output of various OCCs is an important prerequisite for building of a meaningful OCCs ensemble. However, this aspect of the OCC fusion problem has been mostly ignored so far. In this paper, we propose a new method of constructing a fusion of OCCs with three contributions: (a) As a key contribution, enabling an OCC ensemble design using exclusively non anomalous samples, we propose a novel fitness function to evaluate the competency of OCCs without requiring samples from the anomalous class; (b) As a minor, but impactful contribution, we investigate alternative forms of score normalisation of OCCs, and identify a novel two-sided normalisation method as the best in coping with long tail non anomalous data distributions; (c) In the context of building our proposed OCC fusion system based on the weighted averaging approach, we find that the weights optimised using a particle swarm optimisation algorithm produce the most effective solution. We evaluate the merits of the proposed method on 15 benchmarking datasets from different application domains including medical, anti-spam and face spoofing detection. The comparison of the proposed approach with state-of-the-art methods alongside the statistical analysis confirm the effectiveness of the proposed model.