Face Spoofing Research Articles

Explainable biometrics: a systematic literature review

AbstractBiometric systems are largely based on Machine Learning (ML) algorithms which are often considered as a black-box. There is a need to provide them with explanations to make their decision understandable. In this paper, we conduct a Systematic Literature Review aiming at investigating the present adoption of explainable Artificial Intelligence (XAI) techniques in biometric systems. By examining the biometric tasks performed by the selected papers (e.g., face detection or face spoofing), the datasets adopted by the different approaches, the considered ML models, the XAI techniques, and their evaluation methods. We started from 496 papers and, after an accurate analysis, selected 47 papers. Results revealed that XAI is mainly adopted in biometric systems related to the face biometric cues. The explanations provided were all based on model-centric metrics and did not consider how the end-users perceived the explanations, leaving wide space for the biometric researchers to apply the XAI models and enhance the explanation evaluation into an HCI perspective.

Benchmarking Joint Face Spoofing and Forgery Detection With Visual and Physiological Cues

Benchmarking Joint Face Spoofing and Forgery Detection With Visual and Physiological Cues

Saliency-based video summarization for face anti-spoofing

With the growing availability of databases for face presentation attack detection, researchers are increasingly focusing on video-based face anti-spoofing methods that involve hundreds to thousands of images for training the models. However, there is currently no clear consensus on the optimal number of frames in a video to improve face spoofing detection. Inspired by the visual saliency theory, we present a video summarization method for face anti-spoofing detection that aims to enhance the performance and efficiency of deep learning models by leveraging visual saliency. In particular, saliency information is extracted from the differences between the Laplacian and Wiener filter outputs of the source images, enabling the identification of the most visually salient regions within each frame. Subsequently, the source images are decomposed into base and detail images, enhancing the representation of the most important information. Weighting maps are then computed based on the saliency information, indicating the importance of each pixel in the image. By linearly combining the base and detail images using the weighting maps, the method fuses the source images to create a single representative image that summarizes the entire video. The key contribution of the proposed method lies in demonstrating how visual saliency can be used as a data-centric approach to improve the performance and efficiency for face presentation attack detection. By focusing on the most salient images or regions within the images, a more representative and diverse training set can be created, potentially leading to more effective models. To validate the method’s effectiveness, a simple CNN–RNN deep learning architecture was used, and the experimental results showcased state-of-the-art performance on four challenging face anti-spoofing datasets.

Anti Face Spoofing System

Face spoofing, the act of using deceptive techniques such as printed photos or digital images to deceive facial recognition systems, poses a significant threat to the security of various applications, including biometric authentication and access control systems. This paper presents a concise yet effective approach to address the challenge of anti-face spoofing using Python within a limited codebase of 50 lines. The proposed solution leverages a combination of image processing techniques and machine learning algorithms to detect and prevent face spoofing attempts. A pre-trained deep neural network model for facial recognition is employed to extract essential facial features. Subsequently, the system utilizes image manipulation detection methods to identify anomalies indicative of face spoofing attacks. The implementation showcases the simplicity and efficiency of the proposed anti-face spoofing technique, demonstrating the potential for integration into real- world applications with minimal computational overhead. The concise Python code enables easy adoption and adaptation for developers aiming to enhance the security of facial recognition systems against face spoofing threats. The experimental results demonstrate the effectiveness of the approach in accurately distinguishing between genuine and spoofed facial images, thereby contributing to the robustness of facial recognition systems in the presence of adversarial attacks. Keywords - Liveness Detection, Presentation Attack Detection, Spoof Detection Algorithms, Biometric Security, Face Recognition Robustness, 3D Face Modeling, Texture Analysis, Deep Learning Anti- Spoofing, Multi-Spectral Imaging, Behavioral Biometrics, Challenge-Response Mechanisms, Optical Flow Analysis, Pulse Detection, Surface Reflectance, Depth Sensing, Motion Analysis, Pattern Recognition, Anti-Spoofing Networks, Temporal Feature Extraction, Anomaly Detection.

An efficient ensemble explainable AI (XAI) approach for morphed face detection

Numerous deep neural convolutional architectures have been proposed in literature for face Morphing Attack Detection (MADs) to prevent such attacks and lessen the risks associated with them. Although, deep learning models achieved optimal results in terms of performance, it is difficult to understand and analyze these networks since they are black box/opaque in nature. As a consequence, incorrect judgments may be made. There is, however, a dearth of literature that explains decision-making methods of black box deep learning models for biometric Presentation Attack Detection (PADs) or MADs that can aid the biometric community to have trust in deep learning-based biometric systems for identification and authentication in various security applications such as border control, criminal database establishment etc. In this work, we present a novel visual explanation approach named Ensemble XAI integrating Saliency maps, Class Activation Maps (CAM) and Gradient-CAM (Grad-CAM) to provide a more comprehensive visual explanation for a deep learning prognostic model (EfficientNet-B1) that we have employed to predict whether the input presented to a biometric authentication system is morphed or genuine. The experimentations have been performed on three publicly available datasets namely Face Research Lab London (FRLL) dataset, Wide Multi-Channel Presentation Attack (WMCA) dataset, and Makeup Induced Face Spoofing (MIFS) dataset. The experimental evaluations affirms that the resultant visual explanations highlight more fine-grained details of image features/areas focused by EfficientNet-B1 to reach decisions along with appropriate reasoning.

A deep face spoof detection framework using multi-level ELBPs and stacked LSTMs

A deep face spoof detection framework using multi-level ELBPs and stacked LSTMs

Influence of Standalone and Ensemble Classifiers in Face Spoofing Detection using LBP and CNN Models

Background: Artificial intelligence has made significant contributions to facial recognition and biometric identification and is now being employed in a range of applications. Detecting facial spoofing, where someone attempts to pass as an authorized user to gain access to the system, is still difficult. Spoofing-attack-resistant face recognition systems demand efficient and effective solutions. A more stringent recognition system will result in higher false positives and false negatives, which makes such a system questionable for practical use. Eventually, the prominent deep-learning techniques were overtaken by CNN-based architecture. Objective: To analyse classifiers to identify the impact on spoof detection. The intent is not only to get the highest accuracy but also to find strategies to significantly reduce false positives and false negatives. Methods: Face image spoofing detection is implemented in this paper by extracting face embedding using the Local Binary Pattern (LBP) and the VGG16 CNN architecture. To classify real and spoof images, SVM, KNN, Decision Tree, and ensembles of classifier models are utilized. Results: The proposed three models obtained test accuracy of 98%, 94.48%, and 99% when applied to the custom dataset, while in the NUAA photography imposter dataset, they achieved 97%, 99%, and 100% and kept the FN and FP significantly low. Conclusion: Accessing human faces through smart gadgets from various resources is possible, leading to the possibility of spoof attacks. Although spoof detection methods persist, effective methods with high accuracy and low FN and FP are still required. The proposed ensemble techniques significantly outperform the existing classifiers with high accuracy, keeping FN and FP low.

Face and liveness detection with criminal identification using machine learning and image processing techniques for security system

<p>In the past, real-world photos have been used to train classifiers for face liveness identification since the related face presentation attacks (PA) and real-world images have a high degree of overlap. The use of deep convolutional neural networks (CNN) and real-world face photos together to identify the liveness of a face, however, has received very little study. A face recognition system should be able to identify real faces as well as efforts at faking utilizing printed or digital presentations. A true spoofing avoidance method involves observing facial liveness, such as eye blinking and lip movement. However, this strategy is rendered useless when defending against replay assaults that use video. The anti-spoofing technique consists of two modules: the ConvNet classifier module and the blinking eye module, which measure lip and eye movement. The results of the testing demonstrate that the developed module is capable of identifying various face spoof assaults, including those made with the use of posters, masks, or smartphones. To assess the convolutional features in this study adaptively fused from deep CNN produced face pictures and convolutional layers learned from real-world identification. Extensive tests using intra-database and cross-database scenarios on cutting-edge face anti-spoofing databases including CASIA, OULU, NUAA and replay-attack dataset demonstrate that the proposed solution methods for face liveness detection. The algorithm has a 94.30% accuracy rate.</p>

Comparison of Fine-Tuned Networks on Generalization for Face Spoofing Detection

Comparison of Fine-Tuned Networks on Generalization for Face Spoofing Detection

Study of Machine Learning Algorithm based on Face Anti-Spoofing Detection

Face spoofing detection is one of the most well-studied problems in computer vision. Face recognition has become a widely adopted technique in biometric authentication systems. In face recognition based authentication techniques, the system first recognized the person to verify the legitimacy of the user before granting access to the system resources. The system must be able to determine the liveness of the person in front of the camera, for example, by recognizing the face and denying the types of face presentation attacks related to photographs, videos and the 3D mask of the targeted person. Attackers try to directly or indirectly, masquerade the biometric system as another person by forging biometric traits and get unauthorized access. This work studies computer vision- based feature extraction techniques for real and spoof face imaging and combines different features in the area of face anti-spoofing. Keywords: - Face Spoofing, Face Recognition, Machine Learning (ML)

Unveiling Spoofing Attempts: A DCGAN-based Approach to Enhance Face Spoof Detection in Biometric Authentication

Unveiling Spoofing Attempts: A DCGAN-based Approach to Enhance Face Spoof Detection in Biometric Authentication

Enhancing Academic Integrity in Online Assessments: Introducing an Effective Online Exam Proctoring Model using YOLO

The issue of cheating during exams has become a significant concern, gaining widespread attention from the public and media. This issue has been made worse by the transition of educational institutions to an online format, making it simpler to engage in academic dishonesty. To overcome this problem, developed an innovative solution known as the Online Exam Proctoring Model using You Only Look Once (YOLO) Model. With the use of cutting-edge computer vision algorithms, the proposed model makes use of data from cameras and microphones to automatically spot and prevent cheating during online exams. Face orientation, face spoofing detection, mouth tracking, audio analysis, mobile device recognition, and person counting are among the primary elements built into the model. Object detection algorithms, in conjunction with pre-trained YOLO weights, empower the model to detect mobile devices and count the number of individuals present in the exam environment. The automatic termination of exams upon detection of mobile devices, multiple persons, or no persons for 10 seconds acts as a strong deterrent against cheating. The suggested methodology created a reliable and effective tool for online examinations, guaranteeing the academic integrity and enabling impartial assessment of the skills and knowledge of learners. By implementing this solution, educational institutions can foster a secure online exam environment, promoting a culture of honesty and trust between students and educators. As navigate the evolving environment of education, the model stands as a reliable ally in upholding the credibility and authenticity of online learning experiences.

Face spoofing detection using Heterogeneous Auto-Similarities of Characteristics

Facial recognition systems are widely used in various daily applications due to their high accuracy rate and ease of use. Unfortunately, their performance drops drastically when encountering spoofing attacks. In this paper, a novel pipeline for face spoofing detection based on the extraction of Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor from the HSV and YCbCr color spaces is proposed. To detect print and replay spoofing attacks, the HASC descriptor encodes linear and non-linear dependencies between low-level dense features of the color face image using covariance and information-theoretic measures. The combination of covariance and information-theoretic measures makes HASC an efficient descriptor to explore how the features are interrelated in real and fake faces. The HASC texture descriptions are extracted separately from each color channel of the HSV and YCbCr color spaces, and the resulting feature vectors are concatenated into an extended feature vector to better discriminate between real and fake faces. Extensive experiments are conducted on publicly available databases namely, Replay-Attack, CASIA FASD, and MSU MFSD. The obtained results demonstrate that while being of low dimension, the HASC descriptor when fed into a binary Support Vector Machine (SVM) classifier, achieves competitive results compared to the state-of-the-art.

3sXcsNet: A framework for face presentation attack detection using deep learning

Face Recognition is a widely used authentication method deployed for accessing personal digital devices like mobiles and laptops, digital door locks, or various security applications. However, presentation attacks in form of print, video and mask can circumvent such authentication methods. Several solutions based on texture features are proposed in the state-of-the-art; nevertheless, they are sensitive to varying illumination conditions and are less resilient against 2D and 3D attacks. In this paper, we propose a novel 3-stream compact Xception framework (3sXcsNet) consisting of four input streams RGB, MSRCR, MSRCP, and a depth map to form two specific 3-stream fusion (CRF and CPF) to mitigate the illumination scenarios. These streams provide discriminative features. RGB images have rich texture features; MSRCR and MSRCP images are illumination invariant and preserve better chromaticity applied on a color or intensity channel, respectively. In addition, an auxiliary depth map that provides depth information for each pixel makes it easier to analyze features and recognize whether the input image is from a real person or a spoof medium. The 3-stream CRF and CPF represent the features differently and feed independently to a deep CNN compact Xception with a CBAM network. These feature maps are fused using an average fusion mechanism for classification of real and spoof faces. We utilize both 2D and 3D attack datasets namely, REPLAY-ATTACK, CASIA-FASD, OULU-NPU, and 3DMAD to evaluate the effectiveness of the proposed framework. We attain an EER of 1.27% and 0.19% for CRF fusion, as well as 1.48% and 0.20% for CPF fusion when assessing performance on the CASIA-FASD and REPLAY-ATTACK datasets, respectively. On the OULU-NPU dataset, we obtain the most favorable results for two protocols while employing CRF fusion, resulting in ACER metrics of 0.7% and 3.2±2.7%, respectively. On the other hand, CPF fusion outperformed with an ACER of 1.8±1.0% for one protocol. Our findings on intra-database outperform the current state-of-the-art. We also perform cross-database testing to assess the generalization capability of the framework on 2D and 3D attack datasets. Furthermore, the proposed method outperforms both fusion procedures in terms of error rate HTER for intra-database and cross-database testing on the 3DMAD dataset.

Data augmentation based face anti-spoofing (FAS) scheme using deep learning techniques

Recent years have seen a rise in interest in face anti-spoofing (FAS) owing to the critical function it plays in protecting face recognition systems against presentation assaults (PAs). Early-stage FAS approaches relying on handmade characteristics become inaccurate when steadily realistic PAs of unique sorts emerge. Thus, face anti-spoofing algorithms are gaining increasing relevance in such setups. A very innovative method called deep learning has shown remarkable success in difficult computer vision problems. The proposed method uses deep acquisition and transfer of learning to extract characteristics from people’s faces. This is why the authors of this study recommend using the Faster RCNN classifier with a face-liveness detection approach. Two distinct components— the data augmentation module for assessing sparse information as well as the faster RCNN classifier module— make up the anti-spoofing approach. We may use any publicly accessible dataset to train our quicker RCNN classifier. We successively fused these two components and used the Android platform to create a basic face recognition app. The results of the tests demonstrate that the developed module can identify several types of face spoof assaults, such as those carried out with the use of posters, masks, or cell phones. Testing the proposed architecture both across and inside databases using three benchmarking (Idiap Replay Attack, CASIA- FASD, & 3DMAD) demonstrate its ability to deliver outcomes on par with cutting-edge techniques.

A hierarchical multi-modal cross-attention model for face anti-spoofing

Facial recognition has become popular in interactive systems as a means to authenticate identity. However, Facial recognition can be easily attacked illegally through face spoofing. In this paper, we propose a hierarchical multi-modal cross-attention model for face anti-spoofing, which can be flexibly applied in both single-modal and multi-modal scenarios. In order to map features among modalities thoroughly, we also design a novel attention mechanism, namely W-MSA-CA (Window-based Multihead Self-Attention and Cross Attention), which leverages both Multi-modal Multihead Self-Attention (MMSA) and Multi-modal Patch Cross attention (MPCA) to fuse multi-modal features. We test the proposed model on the public datasets and the results show that our model’s capability to detect various types of spoofing is effective.

Analysing texture, color and spatial features for face spoof detection with hybrid classification model

Analysing texture, color and spatial features for face spoof detection with hybrid classification model

Face spoofing detection based on multi-scale color inversion dual-stream convolutional neural network

Face spoofing detection based on multi-scale color inversion dual-stream convolutional neural network

SonarGuard: Ultrasonic Face Liveness Detection on Mobile Devices

Liveness detection has been widely applied in face authentication systems to combat malicious attacks. However, existing methods purely depending on visual frames become vulnerable once visual perception is not reliable. The emerging face spoof and forge techniques urge the systems to exploit the defensive potential of non-visual modalities. To tackle this challenge, we introduce SonarGuard, a system combining ultrasonic and visual information to achieve robust liveness detection on mobile devices. More specifically, SonarGuard simultaneously extracts micro-doppler signatures from ultrasound reflections and motion trajectories from video frames both corresponding to the user’s lip movement. To further confirm the collected ultrasonic and visual information is not derived from malicious audio/video attacks, we consolidate the system via introducing a cross-modal matching mechanism, which demands the inherent consistency between these two modalities. Extensive experiments on a new dataset collected with existing mobile devices demonstrate that the proposed system could achieve average classification error rate of 0.91% under presentation attacks. This result indicates that SonarGuard can boost the security of face authenfication systems in real world usage without additional hardware modification.

Deep learning based intelligent system for robust face spoofing detection using texture feature measurement

The use of biometric structures in our everyday lives is becoming increasingly frequent. Biometrics play a crucial role in various applications, including crime analysis, person identification, and verification. Among different biometrics, the face provides a rich set of features. However, face spoofing is a continuous threat in real-world environments, leading to abnormal behavior in security systems. Conventional face spoofing analysis methods have often failed to achieve optimal performance in detecting face spoofing, faking, and attacks due to limitations in low-level feature extraction. To address this issue, this research introduces a novel technique called Face Spoofing Detection (FSD) based on Deep Learning Convolutional Neural Network (DLCNN), named NLBP-Net. In this technique, features from face images are extracted using Local Binary Pattern (LBP). The periocular area, which remains untouched by the gradient process, is a section of the human face that stands out as being highly unique. Furthermore, the extracted features are trained using the advanced Visual Geometry Group 16 (VGG16) methods. The trained model effectively classifies spoofing and faking attempts in random face images. Simulations conducted on a standard dataset demonstrate that the proposed NLBP-Net outperforms other methods across several metrics. NLBP-Net achieves an accuracy of 99.593%, sensitivity of 99.633%, specificity of 99.137%, recall of 99.224%, precision of 99.057%, and F1-score of 99.057%.