FRGC V2 Database Research Articles

A Hybrid Approach for Face Morphing Detection

Background: In biometrics, one of the most popular study topics is the detection of face morphing attacks. However, because present methods are unable to capture significant feature changes, they are unable to strike the correct balance between accuracy and complexity. Survey investigation and analysis have shown that the existing method of face morphing detection take a bit longer time to detect the image attack due to the high computation required by facial feature extraction approaches. Conversely, further study is needed to develop a model to enhance the computational time and accuracy of the current face morphing recognition methods. The paper developed a hybrid model for face morphing detection. The FERET database was created to aid in the evaluation and development of algorithms. Local Binary Pattern (LBP) was used as feature extraction algorithm and Residue Number System (RNS) was introduced to reduce the lengthy computational time of LBP during the extraction of images. The classification accuracy of 98% was achieved for the FERET database, while an accuracy of 96% was achieved for the FRGCv2 database. An average training time of 0.0532seconds was recorded for the FERET database, while an average training time of 0.0582seconds was achieved for the FRGCv2 database. The study concluded that the high dimensionality of LBP was well reduced and optimized by the RNS algorithm, which improved the performance of face morphing recognition.

Meta-learning-based adversarial training for deep 3D face recognition on point clouds

Recently, deep face recognition using 2D face images has made great advances mainly due to the readily available large-scale face data. However, deep face recognition using 3D face scans, especially on point clouds, has been far from fully explored. In this paper, we propose a novel meta-learning-based adversarial training (MLAT) algorithm for deep 3D face recognition (3DFR) on point clouds. It consists of two alternate modules: adversarial sample generating for 3D face data augmentation and meta-learning-based deep network training. In the first module, adversarial samples of given 3D face scans are dynamically generated based on current deep 3DFR model. In the second module, a meta-learning framework is designed to avoid the performance decrease caused by the generated adversarial samples. Overall, MLAT algorithm combines the adversarial sample generating and meta-learning-based network training in a uniform framework, in which adversarial samples and network parameters are optimized alternately. Thus, it can continuously generate diverse and suitable adversarial samples, and then the meta-learning framework can further improve the accuracy of 3DFR model. Comprehensive experimental results show that the proposed approach consistently achieves competitive rank-one recognition accuracies on the BU-3DFE (100%), Bosphorus (99.78%), BU-4DFE (98.02%) and FRGC v2 (98.01%) database, and thereby substantiate its superiority.

Differential Detection of Facial Retouching: A Multi-Biometric Approach

Facial retouching apps have become common tools which are frequently applied to improve one's facial appearance, e.g. before sharing face images via social media. Beautification induced by retouching has the ability to substantially alter the appearance of face images and hence might represent a challenge for face recognition. Towards deploying secure face recognition as well as enforcing anti-photoshop legislations, a robust and reliable detection of retouched face image is needed. Published approaches consider a single image-based (no-reference) scenario where a potentially retouched face image serves as sole input to the retouching detector. However, in many cases a trusted unaltered face image of a subject examined is available which enables an image pair-based (differential) detection scheme. In this work, ICAO-compliant subsets of the FERET and FRGCv2 face databases are used to automatically create a database containing 9,078 retouched face images together with unconstrained probe images. In evaluations employing the commercial Cognitec FaceVACS and the open-source ArcFace face recognition system, it is shown that facial retouching can negatively impact face recognition performance. Further, a differential facial retouching detection system is proposed which processes pairs of a potentially retouched reference image and corresponding unaltered probe image of single subjects. Estimated differences in feature vectors obtained from texture descriptors, facial landmarks, and deep face representations are leveraged by machine learning-based classifiers of which the detection scores are fused to distinguish between retouched and unaltered face images. The proposed scheme is evaluated in a cross-database scenario where training and testing are performed on the FERET and FRGCv2 databases and vice versa. In the scenario where the used retouching algorithm is known by the detection algorithm, a competitive average D-EER of approximately 2% is achieved. Further, the scenario in which the employed retouching algorithm is not known by the detection algorithm is evaluated. In the latter scenario, the proposed approach obtains an average D-EER below 10% and is shown to outperform several state-of-the-art single image-based detection schemes.

Detection of Face Morphing Attacks Based on PRNU Analysis

Recent research found that attacks based on morphed face images, i.e., morphing attacks, pose a severe security risk to face recognition systems. A reliable morphing attack detection from a single face image remains a research challenge since cameras and morphing techniques used by an attacker are unknown at the time of classification. These issues are commonly overseen while many researchers report encouraging detection performance for training and testing morphing attack detection schemes on images obtained from a single face database employing a single morphing algorithm. In this work, a morphing attack detection system based on the analysis of Photo Response Non-Uniformity (PRNU) is presented. More specifically, spatial and spectral features extracted from PRNU patterns across image cells are analyzed. Differences of these features for bona fide and morphed images are estimated during a threshold-selection stage using the Dresden image database which is specifically built for PRNU analysis in digital image forensics. Cross-database evaluations are then conducted employing an ICAO compliant subset of the FRGCv2 database and a Print-Scan database which is a printed and scanned version of said FRGCv2 subset. Bona fide and morphed face images are automatically generated employing four different morphing algorithms. The proposed PRNU-based morphing attack detector is shown to robustly distinguish bona fide and morphed face images achieving an average D-EER of 11.2% in the best configuration. In scenarios where image sources and morphing techniques are unknown, it is shown to significantly outperform other previously established morphing attack detectors. Finally, the limitations and potential of the approach are demonstrated on a dataset of printed and scanned bona fide and morphed face images.

Dense 3D Face Correspondence.

We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28 mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3 mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5 percent face recognition accuracy on the FRGCv2 and 98.6 percent on Bosphorus database. Our dense model is also able to generalize to unseen datasets.

Robust RGB-D face recognition using Kinect sensor

In this paper we propose a robust face recognition algorithm for low resolution RGB-D Kinect data. Many techniques are proposed for image preprocessing due to the noisy depth data. First, facial symmetry is exploited based on the 3D point cloud to obtain a canonical frontal view image irrespective of the initial pose and then depth data is converted to XYZ normal maps. Secondly, multi-channel Discriminant Transforms are then used to project RGB to DCS (Discriminant Color Space) and normal maps to DNM (Discriminant Normal Maps). Finally, a Multi-channel Robust Sparse Coding method is proposed that codes the multiple channels (DCS or DNM) of a test image as a sparse combination of training samples with different pixel weighting. Weights are calculated dynamically in an iterative process to achieve robustness against variations in pose, illumination, facial expressions and disguise. In contrast to existing techniques, our multi-channel approach is more robust to variations. Reconstruction errors of the test image (DCS and DNM) are normalized and fused to decide its identity. The proposed algorithm is evaluated on four public databases. It achieves 98.4% identification rate on CurtinFaces, a Kinect database with 4784 RGB-D images of 52 subjects. Using a first versus all protocol on the Bosphorus, CASIA and FRGC v2 databases, the proposed algorithm achieves 97.6%, 95.6% and 95.2% identification rates respectively. To the best of our knowledge, these are the highest identification rates reported so far for the first three databases.

Gender and gaze gesture recognition for human-computer interaction

The identification of visual cues in facial images has been widely explored in the broad area of computer vision. However theoretical analyses are often not transformed into widespread assistive Human-Computer Interaction (HCI) systems, due to factors such as inconsistent robustness, low efficiency, large computational expense or strong dependence on complex hardware. We present a novel gender recognition algorithm, a modular eye centre localisation approach and a gaze gesture recognition method, aiming to escalate the intelligence, adaptability and interactivity of HCI systems by combining demographic data (gender) and behavioural data (gaze) to enable development of a range of real-world assistive-technology applications.The gender recognition algorithm utilises Fisher Vectors as facial features which are encoded from low-level local features in facial images. We experimented with four types of low-level features: greyscale values, Local Binary Patterns (LBP), LBP histograms and Scale Invariant Feature Transform (SIFT). The corresponding Fisher Vectors were classified using a linear Support Vector Machine. The algorithm has been tested on the FERET database, the LFW database and the FRGCv2 database, yielding 97.7%, 92.5% and 96.7% accuracy respectively.The eye centre localisation algorithm has a modular approach, following a coarse-to-fine, global-to-regional scheme and utilising isophote and gradient features. A Selective Oriented Gradient filter has been specifically designed to detect and remove strong gradients from eyebrows, eye corners and self-shadows (which sabotage most eye centre localisation methods). The trajectories of the eye centres are then defined as gaze gestures for active HCI. The eye centre localisation algorithm has been compared with 10 other state-of-the-art algorithms with similar functionality and has outperformed them in terms of accuracy while maintaining excellent real-time performance.The above methods have been employed for development of a data recovery system that can be employed for implementation of advanced assistive technology tools. The high accuracy, reliability and real-time performance achieved for attention monitoring, gaze gesture control and recovery of demographic data, can enable the advanced human-robot interaction that is needed for developing systems that can provide assistance with everyday actions, thereby improving the quality of life for the elderly and/or disabled.

Gender recognition from facial images: two or three dimensions?

This paper seeks to compare encoded features from both two-dimensional (2D) and three-dimensional (3D) face images in order to achieve automatic gender recognition with high accuracy and robustness. The Fisher vector encoding method is employed to produce 2D, 3D, and fused features with escalated discriminative power. For 3D face analysis, a two-source photometric stereo (PS) method is introduced that enables 3D surface reconstructions with accurate details as well as desirable efficiency. Moreover, a 2D+3D imaging device, taking the two-source PS method as its core, has been developed that can simultaneously gather color images for 2D evaluations and PS images for 3D analysis. This system inherits the superior reconstruction accuracy from the standard (three or more light) PS method but simplifies the reconstruction algorithm as well as the hardware design by only requiring two light sources. It also offers great potential for facilitating human computer interaction by being accurate, cheap, efficient, and nonintrusive. Ten types of low-level 2D and 3D features have been experimented with and encoded for Fisher vector gender recognition. Evaluations of the Fisher vector encoding method have been performed on the FERET database, Color FERET database, LFW database, and FRGCv2 database, yielding 97.7%, 98.0%, 92.5%, and 96.7% accuracy, respectively. In addition, the comparison of 2D and 3D features has been drawn from a self-collected dataset, which is constructed with the aid of the 2D+3D imaging device in a series of data capture experiments. With a variety of experiments and evaluations, it can be proved that the Fisher vector encoding method outperforms most state-of-the-art gender recognition methods. It has also been observed that 3D features reconstructed by the two-source PS method are able to further boost the Fisher vector gender recognition performance, i.e., up to a 6% increase on the self-collected database.

Beyond parametric score normalisation in biometric verification systems

Similarity scores represent the basis for identity inference in biometric verification systems. However, because of the so-called miss-matched conditions across enrollment and probe samples and identity-dependent factors these scores typically exhibit statistical variations that affect the verification performance of biometric systems. To mitigate these variations, score-normalisation techniques, such as the z-norm, the t-norm or the zt-norm, are commonly adopted. In this study, the authors study the problem of score normalisation in the scope of biometric verification and introduce a new class of non-parametric normalisation techniques, which make no assumptions regarding the shape of the distribution from which the scores are drawn (as the parametric techniques do). Instead, they estimate the shape of the score distribution and use the estimate to map the initial distribution to a common (predefined) distribution. Based on the new class of normalisation techniques they also develop a hybrid normalisation scheme that combines non-parametric and parametric techniques into hybrid two-step procedures. They evaluate the performance of the non-parametric and hybrid techniques in face-verification experiments on the FRGCv2 and SCFace databases and show that the non-parametric techniques outperform their parametric counterparts and that the hybrid procedure is not only feasible, but also retains some desirable characteristics from both the non-parametric and the parametric techniques.

MeshSIFT: Local surface features for 3D face recognition under expression variations and partial data

Matching 3D faces for recognition is a challenging task caused by the presence of expression variations, missing data, and outliers. In this paper the meshSIFT algorithm and its use for 3D face recognition is presented. This algorithm consists of four major components. First, salient points on the 3D facial surface are detected as mean curvature extrema in scale space. Second, orientations are assigned to each of these salient points. Third, the neighbourhood of each salient point is described in a feature vector consisting of concatenated histograms of shape indices and slant angles. Fourth, the feature vectors of two 3D facial surfaces are reliably matched by comparing the angles in feature space. This results in an algorithm which is robust to expression variations, missing data and outliers.As a first contribution, we demonstrate that the number of matching meshSIFT features is a reliable measure for expression-invariant face recognition, as shown by the rank 1 recognition rate of 93.7% and 89.6% for the Bosphorus and FRGC v2 database, respectively. Next, we demonstrate that symmetrising the feature descriptors allows comparing two 3D facial surfaces with limited or no overlap. Validation on the data of the “SHREC’11: Face Scans” contest, containing many partial scans, resulted in a recognition rate of 98.6%, clearly outperforming all other participants in the challenge. Finally, we also demonstrate the use of meshSIFT for two other problems related with 3D face recognition: pose normalisation and symmetry plane estimation. For both problems, applying meshSIFT in combination with RANSAC resulted in a correct solution for ±90% of all Bosphorus database meshes (except ±90° and ±45° rotations).

A Comparative Study of 3-D Face Recognition Under Expression Variations

Research in face recognition has continuously been challenged by extrinsic (head pose, lighting conditions) and intrinsic (facial expression, aging) sources of variability. While many survey papers on face recognition exist, in this paper, we focus on a comparative study of 3-D face recognition under expression variations. As a first contribution, 3-D face databases with expressions are listed, and the most important ones are briefly presented and their complexity is quantified using the iterative closest point (ICP) baseline recognition algorithm. This allows to rank the databases according to their inherent difficulty for face-recognition tasks. This analysis reveals that the FRGC v2 database can be considered as the most challenging because of its size, the presence of expressions and outliers, and the time lapse between the recordings. Therefore, we recommend to use this database as a reference database to evaluate (expression-invariant) 3-D face-recognition algorithms. We also determine and quantify the most important factors that influence the performance. It appears that performance decreases 1) with the degree of nonfrontal pose, 2) for certain expression types, 3) with the magnitude of the expressions, 4) with an increasing number of expressions, and 5) for a higher number of gallery subjects. Future 3-D face-recognition algorithms should be evaluated on the basis of all these factors. As the second contribution, a survey of published 3-D face-recognition methods that deal with expression variations is given. These methods are subdivided into three classes depending on the way the expressions are handled. Region-based methods use expression-stable regions only, while other methods model the expressions either using an isometric or a statistical model. Isometric models assume the deformation because of expression variation to be (locally) isometric, meaning that the deformation preserves lengths along the surface. Statistical models learn how the facial soft tissue deforms during expressions based on a training database with expression labels. Algorithmic performances are evaluated by the comparison of recognition rates for identification and verification. No statistical significant differences in class performance are found between any pair of classes.

Regional Registration for Expression Resistant 3-D Face Recognition

Biometric identification from three-dimensional (3-D) facial surface characteristics has become popular, especially in high security applications. In this paper, we propose a fully automatic expression insensitive 3-D face recognition system. Surface deformations due to facial expressions are a major problem in 3-D face recognition. The proposed approach deals with such challenging conditions in several aspects. First, we employ a fast and accurate region-based registration scheme that uses common region models. These common models make it possible to establish correspondence to all the gallery samples in a single registration pass. Second, we utilize curvature-based 3-D shape descriptors. Last, we apply statistical feature extraction methods. Since all the 3-D facial features are regionally registered to the same generic facial component, subspace construction techniques may be employed. We show that linear discriminant analysis significantly boosts the identification accuracy. We demonstrate the recognition ability of our system using the multiexpression Bosphorus and the most commonly used 3-D face database, Face Recognition Grand Challenge (FRGCv2). Our experimental results show that in both databases we obtain comparable performance to the best rank-1 correct classification rates reported in the literature so far: 98.19% for the Bosphorus and 97.51% for the FRGCv2 database. We have also carried out the standard receiver operating characteristics (ROC III) experiment for the FRGCv2 database. At an FAR of 0.1%, the verification performance was 86.09%. This shows that model-based registration is beneficial in identification scenarios where speed-up is important, whereas for verification one-to-one registration can be more beneficial.

Subspace-Based Holistic Registration for Low-Resolution Facial Images

Subspace-based holistic registration is introduced as an alternative to landmark-based face registration, which has a poor performance on low-resolution images, as obtained in camera surveillance applications. The proposed registration method finds the alignment by maximizing the similarity score between a probe and a gallery image. We use a novel probabilistic framework for both user-independent as well as user-specific face registration. The similarity is calculated using the probability that the face image is correctly aligned in a face subspace, but additionally we take the probability into account that the face is misaligned based on the residual error in the dimensions perpendicular to the face subspace. We perform extensive experiments on the FRGCv2 database to evaluate the impact that the face registration methods have on face recognition. Subspace-based holistic registration on low-resolution images can improve face recognition in comparison with landmark-based registration on high-resolution images. The performance of the tested face recognition methods after subspace-based holistic registration on a low-resolution version of the FRGC database is similar to that after manual registration.

Virtual illumination grid for correction of uncontrolled illumination in facial images

Face recognition under uncontrolled illumination conditions is still considered an unsolved problem. In order to correct for these illumination conditions, we propose a virtual illumination grid (VIG) approach to model the unknown illumination conditions. Furthermore, we use coupled subspace models of both the facial surface and albedo to estimate the face shape. In order to obtain a representation of the face under frontal illumination, we relight the estimated face shape. We show that the frontal illuminated facial images achieve better performance in face recognition. We have performed the challenging Experiment 4 of the FRGCv2 database, which compares uncontrolled probe images to controlled gallery images. Our illumination correction method results in considerably better recognition rates for a number of well-known face recognition methods. By fusing our global illumination correction method with a local illumination correction method, further improvements are achieved.

3D Face Recognition Using Simulated Annealing and the Surface Interpenetration Measure

This paper presents a novel automatic framework to perform 3D face recognition. The proposed method uses a Simulated Annealing-based approach (SA) for range image registration with the Surface Interpenetration Measure (SIM), as similarity measure, in order to match two face images. The authentication score is obtained by combining the SIM values corresponding to the matching of four different face regions: circular and elliptical areas around the nose, forehead, and the entire face region. Then, a modified SA approach is proposed taking advantage of invariant face regions to better handle facial expressions. Comprehensive experiments were performed on the FRGC v2 database, the largest available database of 3D face images composed of 4,007 images with different facial expressions. The experiments simulated both verification and identification systems and the results compared to those reported by state-of-the-art works. By using all of the images in the database, a verification rate of 96.5 percent was achieved at a False Acceptance Rate (FAR) of 0.1 percent. In the identification scenario, a rank-one accuracy of 98.4 percent was achieved. To the best of our knowledge, this is the highest rank-one score ever achieved for the FRGC v2 database when compared to results published in the literature.

Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach

In this paper, we present the computational tools and a hardware prototype for 3D face recognition. Full automation is provided through the use of advanced multistage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact metadata. We present our results on the largest known, and now publicly available, Face Recognition Grand Challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, this is the highest performance reported on the FRGC v2 database for the 3D modality.