Face Recognition Grand Challenge Research Articles

Fusion-Based 2.5D Face Recognition System

Face recognition is the dominant biometrics system used to authenticate an individual’s identity in various applications. Most commercial face recognition systems rely on 2D face images, but the changes in the environment lighting and a person's posture affect the accuracy of the 2D face recognition systems. Hence, the 2.5D face recognition system arises as the solution to eliminate the drawbacks of the 2D face recognition system. The depth feature in the 2.5D data (depth image) provides additional information that can help to improve the accuracy and robustness of 2.5D face recognition systems, particularly in challenging scenarios. This paper proposes a fusion-based approach for the 2.5D face recognition system to enhance the system’s performance, where feature fusion involves the combination of features extracted from the depth image. Gabor-based Region Covariance Matrices (GRCMs) that serve as face identifiers combine the depth and texture images in the structure of a covariance matrix. Several experiments on different fusions have been conducted in the Face Recognition Grand Challenge version 2 (FRGC v2.0) database. This paper shows that the max-min fusion applied to the surface normal (y-direction) and the mean curvature has achieved the best accuracy rate of 93.66% among the other fusion approaches used.

Enhancing eID card mobile-based authentication through 3D facial reconstruction

In recent years the number of credentials and digital identities needed to access online services is exploding, increasing the risk of fraud and unauthorized access to personal data.Biometric verification techniques can help to build more secure authentication systems; in particular face recognition algorithms have reached near perfect accuracy under ideal conditions. In Europe 60% of population owns an electronic identification (eID) document which contains biometric data, and almost all member states provide tools to verify the identity of a user who holds an eID card; however, the adoption of biometric authentication schemes is still limited. Reasons for this low adoption are privacy concerns, about the storage or transmission of personal images, and the degradation of performance of facial recognition algorithms with 2D images acquired "in the wild" (real world condition with inconsistency in lighting and positioning, unclear or occluded facial features, noise from the background). In this paper a new authentication scheme is proposed, exploiting images acquired "in the wild" and 3D facial reconstruction techniques which can help in improving the quality of the probed images, as changes in lighting or position have limited impact on facial shape. Several experiments have been conducted in order to investigate the performance of the proposed method, using the Bosphorus and the NIST Face Recognition Grand Challenge (FRGC) datasets. In particular, one of the most accurate ML face recognition algorithms has been selected to compare the performance of 2D snapshot data, obtained from 3D back-projection, against 2D uncontrolled images acquired "in the wild". We show that using 3D facial reconstructions significantly improve face recognition performance, allowing for a more secure and robust authentication system.

Three-Dimensional Face Recognition Using Solid Harmonic Wavelet Scattering and Homotopy Dictionary Learning.

Data representation has been one of the core topics in 3D graphics and pattern recognition in high-dimensional data. Although the high-resolution geometrical information of a physical object can be well preserved in the form of metrical data, e.g., point clouds/triangular meshes, from a regular data (e.g., image/audio) processing perspective, they also bring excessive noise in the course of feature abstraction and regression. For 3D face recognition, preceding attempts focus on treating the scan samples as signals laying on an underlying discrete surface (mesh) or morphable (statistic) models and by embedding auxiliary information, e.g., texture onto the regularized local planar structure to obtain a superior expressive performance to registration-based methods, but environmental variations such as posture/illumination will dissatisfy the integrity or uniform sampling condition, which holistic models generally rely on. In this paper, a geometric deep learning framework for face recognition is proposed, which merely requires the consumption of raw spatial coordinates. The non-uniformity and non-grid geometric transformations in the course of point cloud face scanning are mitigated by modeling each identity as a stochastic process. Individual face scans are considered realizations, yielding underlying inherent distributions under the appropriate assumption of ergodicity. To accomplish 3D facial recognition, we propose a windowed solid harmonic scattering transform on point cloud face scans to extract the invariant coefficients so that unrelated variations can be encoded into certain components of the scattering domain. With these constructions, a sparse learning network as the semi-supervised classification backbone network can work on reducing intraclass variability. Our framework obtained superior performance to current competing methods; without excluding any fragmentary or severely deformed samples, the rank-1 recognition rate (RR1) achieved was 99.84% on the Face Recognition Grand Challenge (FRGC) v2.0 dataset and 99.90% on the Bosphorus dataset.

Attendance Monitoring System Using Face Recognition

Abstract: The primitive method of taking attendance through pen-paper or registers by the organizations and institutions are not much efficient these days. The proxies of the absentees by their groups or friends are much common due to which this important factor of monitoring the class becomes ineffective. The pen-paper attendance system non-essentially consumes much time in class while smart techniques gives more time to lecturer. It can be simply be manipulated. So, for that many organizations and institutions have replaced it through biometric sensors which mark attendance through fingerprints of students or staff. But due to covid-19, where touching anything is risky. As we know in the current situation during covid-19 pandemic, Government of different nations have a strict guidelines for social distancing required to be followed everywhere keeping the aspect of safety measures in mind. For that, an attendance monitoring system has been devised through face recognition. This face recognition system works in the following stages - face detection, face pre-processing, database creation, face training, face recognition, attendance maintenance. This project spotlight on the significance of the face alignment, hence how precise image is and False Acceptance Rate that can be noticed. The system processes on Face Recognition Grand Challenge (FRGC) with up-to 95% precision. Few students do mark the fake attendance of their classmates by using their digital devices specially phone through which they try to show the picture of their friend to the system but in that case it will display warning message with a beep sound. The motive behind producing face-recognition system is to save the time and make system smart and efficient. Keywords: Face Recognition, Open CV, Machine Learning, Real and fake face detection, Graphical User Interface

Score-Level Fusion of 3D Face and 3D Ear for Multimodal Biometric Human Recognition.

A novel multimodal biometric system is proposed using three-dimensional (3D) face and ear for human recognition. The proposed model overcomes the drawbacks of unimodal biometric systems and solves the 2D biometric problems such as occlusion and illumination. In the proposed model, initially, the principal component analysis (PCA) is utilized for 3D face recognition. Thereafter, the iterative closest point (ICP) is utilized for 3D ear recognition. Finally, the 3D face is fused with a 3D ear using score-level fusion. The simulations are performed on the Face Recognition Grand Challenge database and the University of Notre Dame Collection F database for 3D face and 3D ear datasets, respectively. Experimental results reveal that the proposed model achieves an accuracy of 99.25% using the proposed score-level fusion. Comparative analyses show that the proposed method performs better than other state-of-the-art biometric algorithms in terms of accuracy.

Mesh-based scale-invariant feature transform-like method for three-dimensional face recognition under expressions and missing data

Biometric identification from three-dimensional (3-D) face surface characteristics has become popular. Traditional face recognition methods have achieved very high recognition accuracy under controlled environments. However, 3-D face recognition technology still faces a great challenge for facial expressions and missing data caused by pose variations or occlusions. The mesh-based scale-invariant feature transform (SIFT)-like method is one of the most effective methods to deal with these issues. However, eliminating edge response has not been considered in this kind of method, which affects recognition performance and computation cost. To address this problem, a mesh edge point filter is proposed to remove edge keypoints. For feature description, a local descriptor that is composed of the histogram of geometric shapes and the histogram of shape index is designed to describe local shapes of keypoints. A complete mesh-based SIFT-like framework for 3-D face recognition is also presented. Results from experiments based on the Bosphorus and face recognition grand challenge v2.0 databases show that the proposed method is effective for expression variations, occlusions, and pose variations. Compared with several state-of-the-art methods, the proposed method has obtained a great compromise between computational complexity and recognition performance.

A multi-scale three-dimensional face recognition approach with sparse representation-based classifier and fusion of local covariance descriptors

In this paper, an efficient multi-scale hybrid approach is proposed to tackle two main problems in three-dimensional (3D) face recognition, namely the singularity of scale features representation and underexplored locality in dictionary learning. The multi-scale features space representation is developed based on the new 3D faces generated by the Gaussian filter. The locality-sensitive Riemannian sparse representation-based classifier is also constructed to accurately recognize faces with various expressions, poses and occlusions. Two sets of face recognition experiment, one that includes expression variations, and the another that includes pose and occlusion variations, are conducted to compare the performance of the proposed approach against other benchmark 3D face recognition algorithms. The recognition accuracies of the proposed algorithm to both Neutral vs. Neutral achieved on Face Recognition Grand Challenge (FRGC) v2.0 database and Bosphorus database are 100%.

3D Security User Identification in Banking

Biometric systems may be divided into 2 categories betting on the characteristics used. One category uses physical characteristics that area unit associated with the form and presence of the body and body components, like fingerprint, finger knuckles, face (2-D and 3-D), DNA, hand and palm pure mathematics, iris texture, and retinal vasculature. Systems belong to the second category use activity characteristics, like gait, handwriting, keyboard writing, and speech. Analysis in face recognition has endlessly been challenged by alien (head create, lighting conditions) and intrinsic (facial expression, aging) sources of variability. During this system is employed to several organizations and lots of applications for security purpose. Many approaches area unit face recognition exists, during this project, specialize in a comparative study of 3D face recognition beneath expression variations. First 3D face databases with expressions area unit listed, and therefore the most significant ones area unit conferred and their complexness is quantified victimisation principal part analysis, linear discriminate analysis and native binary patterns. The project to be real time enforced datasets to reason the assorted varieties of expressions. Pictures in terms of the popularity performance are evaluated with 3 completely different techniques (principal component analysis, linear discriminant analysis, and native binary patterns) on face recognition grand challenge and strait 3D face databases.

A modified technique for face recognition under degraded conditions

In this paper an improved face recognition algorithm under degrading conditions is proposed. The proposed algorithm uses a combination of preprocessing techniques coupled with discriminative feature extractors to obtain the best distinctive features for classification. Preprocessing approach is the fusion of multi-scale Weber and enhanced complex wavelet transform. Combination of multiple feature extraction based on Gabor filters, block-based local phase quantization (LPQ) coupled with principal component analysis (PCA) proved to be very effective to improve correct rate of recognition. We have also used two known classifiers, extreme learning machine (ELM), and sparse classifier (SC), and fused their outputs to obtain best recognition rate. Experimental results show improved performance of proposed algorithm under poor illumination, partial occlusion and low-quality images in uncontrolled conditions. Our best recognition results using second version of face recognition grand challenge (FRGC 2.0.4) which is the most challenging database, indicated more than 28% improvement over previous works.

Score‐level fusion using generalized extreme value distribution and DSmT, for multi‐biometric systems

Human recognition in a multi-biometric system is performed by combining biometric clues from different sources (multiple sensors, units, algorithms, samples and modalities) at different levels (sensor, feature, score, rank and decision level). Low computational complexity and adequate data for fusion make the score-level fusion a preferable option over other levels of fusion. However, incompatibility issue prevails at this level as scores obtained from different uni-biometric systems are disparate in nature. This disparity can be resolved by using score normalisation before fusion. This study first analysed the effect of generalised extreme value distribution-based score normalisation technique on different fusion techniques and then proposes an efficient score fusion technique based on Dezert-Smarandache theory (DSmT). A unique blend of belief assignment and decision-making methods in the DSmT framework is proposed for score-level fusion. For evaluation of the proposed method, experiments are performed on multi-algorithm, multi-unit and multi-modal biometrics systems created from three publicly available datasets: (i) NIST BSSR1 multi-biometric score database, (ii) face recognition grand challenge (FRGC) V2.0 and (iii) LG4000 iris dataset. Comparative studies of performance analysis show the efficiency of the authors' proposed method over other recently published state-of-the-art fusion methods.

Within‐component and between‐component multi‐kernel discriminating correlation analysis for colour face recognition

The key problem of colour face recognition technique is how to take full advantage of the colour information and extract effective discriminating features. To solve this problem, the authors propose a novel non-linear feature extraction approach for colour face recognition, named dual multi-kernel discriminating correlation analysis, which separately maps different colour components of face images into different non-linear kernel spaces, and then implements multi-kernel learning and discriminant analysis with the correlation metric not only within each colour component but also between diverse components. Then, to choose the optimum kernel space for each colour component and select the most suitable colour space for their approach, they design a kernel selection strategy and a colour space selection strategy, respectively. Experimental results in the face recognition grand challenge version 2 and labelled faces in the wilds databases validate the effectiveness of the proposed approach and two strategies.

Robust local features for remote face recognition

In this paper, we propose a robust local descriptor for face recognition. It consists of two components, one based on a shearlet-decomposition and the other on local binary pattern (LBP). Shearlets can completely analyze the singular structures of piecewise smooth images, which is useful since singularities and irregular structures carry useful information in an underlying image. Furthermore, LBP is effective for describing the edges extracted by shearlets even when the images contain high level of noise. Experimental results using the Face Recognition Grand Challenge dataset show that the proposed local descriptor significantly outperforms many widely used features (e.g., Gabor and deep learning-based features) when the images are corrupted by random noise, demonstrating robustness to noise. In addition, experimental results show promising results for two challenging datasets which have poor image quality, i.e., a remote face dataset and the Point and Shoot Face Recognition Challenge dataset.

Dual multi-kernel discriminant analysis for color face recognition

With the increasing use of color images in the fields of pattern recognition, computer vision and machine learning, color face recognition technique becomes important, whose key problem is how to make full use of the color information and extract effective discriminating features. In this paper, we propose a novel nonlinear feature extraction approach for color face recognition, named dual multi-kernel discriminant analysis (DMDA), where we design a kernel selection strategy to select the optimal kernel mapping function for each color component of face images, further design a color space selection strategy to choose the most suitable space, then separately map different color components of face images into different high-dimensional kernel spaces, and finally perform multi-kernel learning and discriminant analysis not only within each component but also between different components. Experimental results in the public face recognition grand challenge (FRGC) version 2 and labeled faces in the wilds (LFW) databases illustrate that our approach outperforms several representative color face recognition methods.

Elliptical Higher-Order-Spectra Periocular Code

The periocular region has recently emerged as a standalone biometric trait, promising attractive tradeoff between the iris alone and the entire face, especially for cases where neither the iris nor a full facial image can be acquired. This advantage provides another dimension for implementing a robust biometric system performed in non-ideal conditions. Global features [local binary pattern (LBP), Histogram of Gradient (HOG)] and local features have been introduced; however, the performance of these features can deteriorate for images captured in unconstrained and less-cooperative conditions. A particular set of higher order spectral (HOS) features have been proved to be invariant to translation, scale, rotation, brightness level shift, and contrast change. These properties are desirable in the periocular recognition problem to deal with the non-ideal imaging conditions. This paper investigates the HOS features in different configurations for the periocular recognition problem under non-ideal conditions. Specifically, we introduce a new sampling approach for the periocular region based on an elliptical coordinate. This non-linear sampling approach is then combined with the robustness of the HOS features for encoding the periocular region. In addition, we also propose a new technique for combining left and right perioculars. The proposed feature-level fusion approach is based on the state-of-the-art bilinear pooling technique to allow efficient interaction between the features of both perioculars. We show the validity of the proposed approach in encoding discriminant features outperforming or comparing favorably with the state-of-the-art features on the two popular data sets: Face Recognition Grand Challenge and Japanese Female Facial Expression.

Sparse 3D directional vertices vs continuous 3D curves: Efficient 3D surface matching and its application for single model face recognition

Traditionally, point clouds and meshes are used to represent and match 3D shapes, which often cannot meet the computational speed and storage space requirements in many 3D data matching and retrieval applications. In this paper, we present a novel 3D directional vertices (3D2V) approach to efficiently represent and match 3D surfaces by much fewer sparsely distributed structured vertices that carry structural information transferred from their deleted neighbouring points. A 3D2V conversion and similarity measurement method is developed to compute the distance between two different 3D2Vs. The performance of the proposed method is evaluated on 3D face recognition using Face Recognition Grand Challenge v2.0 (FRGC v2.0) and GavabDB databases and compared with the curve-based benchmark method. The experimental results demonstrate that the proposed 3D2V method can significantly reduce the data storage requirement and computation time with a moderate increase of accuracy at the same time. It provides a new tool for developing fast 3D surface matching algorithms for large scale 3D data classification and retrieval.

Automatic landmark detection and 3D Face data extraction

This paper contributes to 3D facial synthesis by presenting a novel method for parameterization using Landmark Point detection. The approach presented aims at improving facial recognition even in varying facial expressions, and missing data in 3D facial models. As such, the prime objective was to develop an automatically embedded process that can detect any frontal face in 3D face recognition systems, with face segmentation and surface curvature information. Using the hybrid interpolation method, experiments on facial landmarks were performed on 4950 images from Face Recognition Grand Challenge database (FRGC). Distinctive facial landmarks from the nose–tips, Limits mouth and two eye corners formed the statistical inputs for Iterative Closest Point (ICP) in the Point Distribution Model (PDM). Performance or landmark localization is reported by using percentage deviation from the mean 3D profile. Localization results and estimated data on landmark locations demonstrate that the method confirms its effectiveness for proposed application.

Multimodal 2D–3D face recognition using local descriptors: pyramidal shape map and structural context

In this study, the authors propose a local descriptor based multimodal approach to improve face recognition performance. Pre-processing is done to smooth, resample, and register data. The resampled three-dimensional (3D) face data are applied to extract novel descriptors including pyramidal shape index, pyramidal curvedness, pyramidal mean, and Gaussian curvatures. Proposed pyramidal shape maps are extracted at each level of the Gaussian pyramid on each point of the 3D data to have 2D matrices as representatives of 3D geometry information. A local descriptor structural context histogram, which represents the structure of the image using scale invariant feature transform, is calculated on pyramidal shape map descriptors and texture image to find matched keypoints in 3D and 2D modality, respectively. Score-level fusion by means of sum rule is employed to get a final matching score. Experimental results on the Face Recognition Grand Challenge (FRGC v2) database illustrate verification rates of 99 and 98.65% at 0.1% false acceptance rate for all versus all and ROC III experiments, respectively. On Bosphorus database, verification rate of 95.8% for neutral versus all experiment has been achieved.

Recognizing faces prone to occlusions and common variations using optimal face subgraphs

An intuitive graph optimization face recognition approach called Harmony Search Oriented-EBGM (HSO-EBGM) inspired by the classical Elastic Bunch Graph Matching (EBGM) graphical model is proposed in this contribution. In the proposed HSO-EBGM, a recent evolutionary approach called harmony search optimization is tailored to automatically determine optimal facial landmarks. A novel notion of face subgraphs have been formulated with the aid of these automated landmarks that maximizes the similarity entailed by the subgraphs. For experimental evaluation, two sets of de facto databases (i.e., AR and Face Recognition Grand Challenge (FRGC) ver2.0) are used to validate and analyze the behavior of the proposed HSO-EBGM in terms of number of subgraphs, varying occlusion sizes, face images under controlled/ideal conditions, realistic partial occlusions, expression variations and varying illumination conditions. For a number of experiments, results justify that the HSO-EBGM shows improved recognition performance when compared to recent state-of-the-art face recognition approaches.

Coupled Learning for Facial Deblur.

Blur in facial images significantly impedes the efficiency of recognition approaches. However, most existing blind deconvolution methods cannot generate satisfactory results due to their dependence on strong edges, which are sufficient in natural images but not in facial images. In this paper, we represent point spread functions (PSFs) by the linear combination of a set of pre-defined orthogonal PSFs, and similarly, an estimated intrinsic (EI) sharp face image is represented by the linear combination of a set of pre-defined orthogonal face images. In doing so, PSF and EI estimation is simplified to discovering two sets of linear combination coefficients, which are simultaneously found by our proposed coupled learning algorithm. To make our method robust to different types of blurry face images, we generate several candidate PSFs and EIs for a test image, and then, a non-blind deconvolution method is adopted to generate more EIs by those candidate PSFs. Finally, we deploy a blind image quality assessment metric to automatically select the optimal EI. Thorough experiments on the facial recognition technology database, extended Yale face database B, CMU pose, illumination, and expression (PIE) database, and face recognition grand challenge database version 2.0 demonstrate that the proposed approach effectively restores intrinsic sharp face images and, consequently, improves the performance of face recognition.

Bayesian face recognition using 2D Gaussian-Hermite moments

This paper presents a statistical face recognition algorithm by expressing face images in terms of orthogonal two-dimensional Gaussian-Hermite moments (2D-GHMs). Motivation for developing 2D-GHM-based recognition algorithm includes the ability of these moments to capture higher-order hidden nonlinear 2D structures within images and the invariance of certain linear combinations of moments to common geometric distortions in images. The key contribution of this paper is that features of 2D faces are represented in terms of a statistically selected set of 2D discriminative GHMs (DGHMs) as opposed to commonly chosen heuristic set of first few order moments only. In particular, the intraclass correlation coefficient for the entire set of moments of the training images are used to select only a desired set of moments that maximize the discrimination among available classes. The naive Bayes classifier that yields optimal performance in many statistical applications is used for identification due to the simplicity of its implementation for handling huge size face database. Experiments are conducted to evaluate the performance of the proposed recognition algorithm on exhaustive databases such as the AT&T, Face Recognition Grand Challenge (FRGC), Face Recognition Technology (FERET), Labeled Faces in the Wild (LFW), and YouTube that possess face images or videos with significant variations in terms of appearance, occlusion, expression, pose, resolution, and illumination both in the constrained and unconstrained environments. In the constrained condition, comparisons with the well-established 2D-principal component analysis, 2D-linear discriminant analysis, and 2D-canonical correlation analysis methods as well as orthogonal 2D-Krawtchouk moment-based method reveal the superior performance of the proposed method in terms of recognition accuracy for varying numbers of training and probe images. The proposed DGHM features also show superior recognition or verification performance on the standard protocols of the unconstrained face databases when comparing with the commonly referred descriptors such as the local binary pattern or scale-invariant feature transform.