EResNeXt: A Multiple Representation and Attention Based Technique for Fingerprint Presentation Attack Detection in Unknown Presentation Attack Instruments Scenario

The increased need for unattended authentication in multiple scenarios has motivated a wide deployment of biometric systems in the last few years. This has in turn led to the disclosure of security concerns specifically related to biometric systems. Among them, Presentation Attacks (PAs, i.e., attempts to log into the system with a fake biometric characteristic or presentation attack instrument) pose a severe threat to the security of the system: any person could eventually fabricate or order a gummy finger or face mask to impersonate someone else. The biometrics community has thus made a considerable effort to the development of automatic Presentation Attack Detection (PAD) mechanisms, for instance through the international LivDet competitions. In this context, we present a novel fingerprint PAD scheme based on $i)$ a new capture device able to acquire images within the short wave infrared (SWIR) spectrum, and $ii)$ an in-depth analysis of several state-of-the-art techniques based on both handcrafted and deep learning features. The approach is evaluated on a database comprising over 4700 samples, stemming from 562 different subjects and 35 different presentation attack instrument (PAI) species. The results show the soundness of the proposed approach with a detection equal error rate (D-EER) as low as 1.36\% even in a realistic scenario where five different PAI species are considered only for testing purposes (i.e., unknown attacks).

The main scope of this chapter is to serve as an introduction to face presentation attack detection, including key resources and advances in the field in the last few years. The next pages present the different presentation attacks that a face recognition system can confront, in which an attacker presents to the sensor, mainly a camera, a Presentation Attack Instrument (PAI), that is generally a photograph, a video, or a mask, with the target to impersonate a genuine user or to hide the actual identity of the attacker via obfuscation. First, we make an introduction of the current status of face recognition, its level of deployment, and its challenges. In addition, we present the vulnerabilities and the possible attacks that a face recognition system may be exposed to, showing that way the high importance of presentation attack detection methods. We review different types of presentation attack methods, from simpler to more complex ones, and in which cases they could be effective. Then, we summarize the most popular presentation attack detection methods to deal with these attacks. Finally, we introduce public datasets used by the research community for exploring vulnerabilities of face biometrics to presentation attacks and developing effective countermeasures against known PAIs.

Face presentation attacks are becoming more efficient since new 3D facial masks are used. Passive terahertz imaging offers specific physical properties that may improve presentation attack detection capabilities. The non-zero transmission capability through a variety of non-metallic materials may provide necessary information for presentation attack detection. The aim of this paper is to present outcomes of a study on face presentation attack detection using passive imaging at 250 GHz. An analysis of presentation attacks for facial recognition systems using custom displayed and printed photographs, 3D-printed and full-face flexible 3D-latex masks, is provided together with spectral characterization of various presentation attack instruments. A set of experiments with various instruments and various sets of clothing is described and discussed. Finally, two presentation attack detection methods are proposed. The first method is based on a threshold corresponding to a difference between mean intensities of selected regions of interests while the second method uses eight different deep learning classifiers to detect presentation attacks. Results of two validation schemes are presented.

As face presentation attacks (PAs) are realistic threats for unattended face verification systems, face presentation attack detection (PAD) has been intensively investigated in past years, and the recent advances in face PAD have significantly reduced the success rate of such attacks. In this article, an empirical study on a novel and effective face impostor PA is made. In the proposed PA, a facial artifact is created by using the most vulnerable facial components, which are optimally selected based on the vulnerability analysis of different facial components to impostor PAs. An attacker can launch a face PA by presenting a facial artifact on his or her own real face. With a collected PA database containing various types of artifacts and presentation attack instruments (PAIs), the experimental results and analysis show that the proposed PA poses a more serious threat to face verification and PAD systems compared with the print, replay, and mask PAs. Moreover, the generalization ability of the proposed PA and the vulnerability analysis with regard to commercial systems are also investigated by evaluating unknown face verification and real-world PAD systems. It provides a new paradigm for the study of face PAs.

Nowadays, fingerprint-based biometric recognition systems are becoming increasingly popular. However, in spite of their numerous advantages, biometric capture devices are usually exposed to the public and thus vulnerable to presentation attacks (PAs). Therefore, presentation attack detection (PAD) methods are of utmost importance in order to distinguish between bona fide and attack presentations. Owing to the nearly unlimited possibilities to create new presentation attack instruments (PAIs), unknown attacks are a threat to the existing PAD algorithms. This fact motivates research on generalisation capabilities in order to find PAD methods that are resilient to new attacks. In this context, the authors evaluate the generalisability of multiple PAD algorithms on a dataset of 19,711 bona fide and 4339 PA samples, including 45 different PAI species. The PAD data is captured in the short wave infrared domain, and the results discuss the advantages and drawbacks of this PAD technique regarding unknown attacks.

Due to the wide operational deployment of biometric recognition systems, presentation attacks targeting the capture device have become a severe threat. Especially for fingerprint recognition, a high number of different materials allows the creation of numerous presentation attack instruments (PAIs) in the form of full fake fingers and fingerprint overlays, which very much resemble the skin properties at fingertips. As a consequence, automated presentation attack detection (PAD) mechanisms are of utmost importance. Utilising a 1310 nm laser in a new capture device, we present an evaluation of three long short-term memory (LSTM) networks in comparison to eight convolutional neural networks (CNNs) on a database comprising over 22,000 samples and including 45 different PAI species. The LSTMs analyse temporal properties within a captured sequence in order to detect blood movement, while the CNNs take into account spatial properties within a single frame to focus on reflections by the PAI material. The results show that the diversity of PAI species is too big for a single classifier to correctly detect all presentation attacks. However, by fusing the scores from distinct algorithms, we can achieve a detection accuracy of 3.71% APCER for a convenient BPCER of 0.2%.

Face recognition systems face real challenges from various presentation attacks. New, more sophisticated methods of presentation attacks are becoming more difficult to detect using traditional face recognition systems. Thermal infrared imaging offers specific physical properties that may boost presentation attack detection capabilities. The aim of this paper is to present outcomes of investigations on the detection of various face presentation attacks in thermal infrared in various conditions including thermal heating of masks and various states of subjects. A thorough analysis of presentation attacks using printed and displayed facial photographs, 3D-printed, custom flexible 3D-latex and silicone masks is provided. The paper presents the intensity analysis of thermal energy distribution for specific facial landmarks during long-lasting experiments. Thermalization impact, as well as varying the subject’s state due to physical effort on presentation attack detection are investigated. A new thermal face spoofing dataset is introduced. Finally, a two-step deep learning-based method for the detection of presentation attacks is presented. Validation results of a set of deep learning methods across various presentation attack instruments are presented.

FinPAD: State-of-the-art of fingerprint presentation attack detection mechanisms, taxonomy and future perspectives

The LivDet‐2020 competition focuses on Presentation Attacks Detection (PAD) algorithms, has still open problems, mainly unknown attack scenarios. It is crucial to enhance PAD methods. This can be achieved by augmenting the number of Presentation Attack Instruments (PAI) and Bona fide (genuine) images used to train such algorithms. Unfortunately, the capture and creation of PAI and even the capture of Bona fide images are sometimes complex to achieve. The generation of synthetic images with Generative Adversarial Networks (GAN) algorithms may help and has shown significant improvements in recent years. This paper presents a benchmark of GAN methods to achieve a novel synthetic PAI from a small set of periocular near‐infrared images. The best PAI was obtained using StyleGAN2, and it was tested using the best PAD algorithm from the LivDet‐2020. The synthetic PAI was able to fool such an algorithm. As a result, all images were classified as Bona fide. A MobileNetV2 was trained using the synthetic PAI as a new class to achieve a more robust PAD. The resulting PAD was able to classify 96.7% of synthetic images as attacks. BPCER 10 was 0.24%. Such results demonstrated the need for PAD algorithms to be constantly updated and trained with synthetic images.

Presentation attacks (a.k.a, direct attacks or spoofing attacks) against face recognition systems have emerged as a serious security threat. To mitigate these attacks on conventional face recognition systems, several Presentation Attack Detection (PAD) algorithms have been developed, which address various Presentation Attack Instruments (PAI) including 3D face masks, 2D photo, wrap photo and electronic display, that can be used for the attack. In this paper, we demonstrate and evaluate the vulnerability of an extended Multispectral face recognition system. The extended Multispectral system captures the face image across various spectral bands, thus, we propose to study each of these spectral bands for the vulnerability towards presentation attacks. We have employed a commercial Multispectral camera - SpectraCam™ that can capture seven different spectral bands to collect both bona-fide (a.k.a, live or normal or real) samples as well as artefact (or spoof) face samples. Extensive experiments are carried out on the newly compiled database to provide insights on the vulnerability of the extended Multispectral face system towards PAI generated using a printer. We have created the face artefacts using two different printers, which include laser and inkjet printers. Further, we have also evaluated the state-of-the-art PAD algorithms that are widely employed in conventional face PAD systems. Our study reveals the vulnerability of extended Multispectral face recognition system with respect to the print attack. The results obtained using state-of-the-art PAD algorithms further indicate the challenge to detect the presentation attacks in extended Multispectral face recognition systems.

The robustness and generalization ability of Presentation Attack Detection (PAD) methods is critical to ensure the security of Face Recognition Systems (FRSs). However, in a real scenario, Presentation Attacks (PAs) are various and it is hard to predict the Presentation Attack Instrument (PAI) species that will be used by the attacker. Existing PAD methods are highly dependent on the limited training set and cannot generalize well to unknown PAI species. Unlike this specific PAD task, other face related tasks trained by huge amount of real faces (e.g. face recognition and attribute editing) can be effectively adopted into different application scenarios. Inspired by this, we propose to trade position of PAD and face related work in a face system and apply the free acquired prior knowledge from face related tasks to solve face PAD, so as to improve the generalization ability in detecting PAs. The proposed method, first introduces task specific features from other face related task, then, we design a Cross-Modal Adapter using a Graph Attention Network (GAT) to re-map such features to adapt to PAD task. Finally, face PAD is achieved by using the hierarchical features from a CNN-based PA detector and the re-mapped features. The experimental results show that the proposed method can achieve significant improvements in the complicated and hybrid datasets, when compared with the state-of-the-art methods. In particular, when training on the datasets OULU-NPU, CASIA-FASD, and Idiap Replay-Attack, we obtain HTER (Half Total Error Rate) of 5.48% for the testing dataset MSU-MFSD, outperforming the baseline by 7.39%. The source code is available at https://github.com/WentianZhang-ML/FRT-PAD.KeywordsFacePresentation attack detectionGraph neural network

In recent years, the popularity of fingerprint-based biometric authentication systems significantly increased. However, together with many advantages, biometric systems are still vulnerable to presentation attacks (PAs). In particular, this applies for unsupervised applications, where new attacks unknown to the system operator may occur. Therefore, presentation attack detection (PAD) methods are used to determine whether samples stem from a bona fide subject or from a presentation attack instrument (PAI). In this context, most works are dedicated to solve PAD as a two-class classification problem, which includes training a model on both bona fide and PA samples. In spite of the good detection rates reported, these methods still face difficulties detecting PAIs from unknown materials. To address this issue, we propose a new PAD technique based on autoencoders (AEs) trained only on bona fide samples (i.e. one-class), which are captured in the short wave infrared domain. On the experimental evaluation over a database of 19,711 bona fide and 4,339 PA images including 45 different PAI species, a detection equal error rate (D-EER) of 2.00% was achieved. Additionally, our best performing AE model is compared to further one-class classifiers (support vector machine, Gaussian mixture model). The results show the effectiveness of the AE model as it significantly outperforms the previously proposed methods.

Fingerprint-based biometric systems have experienced a large development in the past. In spite of many advantages, they are still vulnerable to attack presentations (APs). Therefore, the task of determining whether a sample stems from a live subject (i.e., bona fide) or from an artificial replica is a mandatory requirement which has recently received a considerable attention. Nowadays, when the materials for the fabrication of the Presentation Attack Instruments (PAIs) have been used to train the Presentation Attack Detection (PAD) methods, the PAIs can be successfully identified in most cases. However, current PAD methods still face difficulties detecting PAIs built from unknown materials and/or unknown recepies, or acquired using different capture devices. To tackle this issue, we propose a new PAD technique based on three image representation approaches combining local and global information of the fingerprint. By transforming these representations into a common feature space, we can correctly discriminate bona fide from attack presentations in the aforementioned scenarios. The experimental evaluation of our proposal over the LivDet 2011 to 2019 databases, yielded error rates outperforming the top state-of-the-art results by up to 72% in the most challenging scenarios. In addition, the best representation achieved the best results in the LivDet 2019 competition (overall accuracy of 96.17%).

State-of-the-art spoof detection methods tend to overfit to the spoof types seen during training and fail to generalize to unknown spoof types. Given that face anti-spoofing is inherently a local task, we propose a face anti-spoofing framework, namely Self-Supervised Regional Fully Convolutional Network (SSR-FCN), that is trained to learn local discriminative cues from a face image in a self-supervised manner. The proposed framework improves generalizability while maintaining the computational efficiency of holistic face anti-spoofing approaches (< 4 ms on a Nvidia GTX 1080Ti GPU). The proposed method is interpretable since it localizes which parts of the face are labeled as spoofs. Experimental results show that SSR-FCN can achieve TDR = 65% @ 2.0% FDR when evaluated on a dataset comprising of 13 different spoof types under unknown attacks while achieving competitive performances under standard benchmark datasets (Oulu-NPU, CASIA-MFSD, and Replay-Attack).

Contactless fingerprint recognition is known for its high user comfort and low hygienic concerns. However, contact-less fingerprint recognition, especially in mobile and un-supervised scenarios, is vulnerable to presentation attacks. Presentation Attack Detection (PAD) in biometric systems like contactless fingerprint recognition is more challenging compared with contact-based modalities because many de-tection mechanisms rely on direct contact between the finger and the surface of the capture device. Hence, in contactless scenarios it is generally possible to present more Presentation Attack Instruments (PAIs) like printout or replay arte-facts. In this work, we introduce COLFISPOOF, a new database for contactless fingerprint PAD. The database is acquired using a contactless fingerprint recognition system utilizing a smartphone as capturing device. It comprises 7,200 samples of 72 different PAI species and was captured with two different smartphone models. The database is publicly available for research purposes such that interested researchers can download and use it to develop new PAD algorithms. Moreover, we define evaluation protocols for training and testing of machine learning algorithms such that future PAD algorithms can be benchmarked on this database in a comparable and reproducible way.