Pose Invariant Face Recognition Research Articles

Performance Analysis of Pose Invariant Face Recognition Approaches in Unconstrained Environments

Face recognition and authentication are two significant and dynamic research issues in computer vision applications. There are many factors that should be accounted for face recognition; among them pose variation is a major challenge which severely influence in the performance of face recognition. In order to improve the performance, several research methods have been developed to perform the face recognition process with pose invariant conditions in constrained and unconstrained environments. In this paper, the authors analyzed the performance of a popular texture descriptors viz., Local Binary Pattern, Local Derivative Pattern and Histograms of Oriented Gradients for pose invariant problem. State of the art preprocessing techniques such as Discrete Cosine Transform, Difference of Gaussian, Multi Scale Retinex and Gradient face have also been applied before feature extraction. In the recognition phase K- nearest neighbor classifier is used to accomplish the classification task. To evaluate the efficiency of pose invariant face recognition algorithm three publicly available databases viz. UMIST, ORL and LFW datasets have been used. The above said databases have very wide pose variations and it is proved that the state of the art method is efficient only in constrained situations.

Pose‐invariant face recognition using curvelet neural network

A novel pose-invariant face recognition method is proposed by combining curvelet-invariant moments with curvelet neural network. First a special set of statistical coefficients using higher-order moments of curvelet are extracted as the feature vector and then the invariant features are fed into curvelet neural networks. Finally, supervised invariant face recognition is achieved by converging the neural network using curvelet as the activation function of the hidden layer neurons. The experimental results demonstrate that curvelet higher-order moments and curvelet neural networks achieve higher accuracy for face recognition across pose and converge rapidly than standard back propagation neural networks.

Fast pose invariant face recognition using super coupled multiresolution Markov Random Fields on a GPU

We discuss the problem of pose invariant face recognition using a Markov Random Field (MRF) model. MRF image to image matching has been shown to be very promising in earlier studies (Arashloo and Kittler, 2011) [4]. Its demanding computational complexity has been addressed in Arashloo et al. (2011) [6] by means of multiresolution MRFs linked by the super coupling transform advocated by Petrou et al. (1998) [37, 11]. In this paper, we benefit from the daisy descriptor for face image representation in image matching. Most importantly, we design an innovative GPU implementation of the proposed multiresolution MRF matching process. The significant speed up achieved (factor of 25) has multiple benefits: It makes the MRF approach a practical proposition. It facilitates extensive empirical optimisation and evaluation studies. The latter conducted on benchmarking databases, including the challenging labelled faces in the wild (LFW) database show the outstanding potential of the proposed method, which consistently achieves state-of-the-art performance in standard benchmarking tests. The experimental studies also show that the super coupled multiresolution MRFs deliver a computational speed up by a factor of 5 over and above the speed up achieved using the GPU implementation.

Rough set based pose invariant face recognition with mug shot images

This paper presents a pose invariant face recognition system to annihilate pose problem by using a modified log Gabor algorithm and the concepts of rough sets together. The proposed system consists of four separate modules; face detection, pose classification, feature extraction and recognition. Rough membership function rmf classifier has been used to classify the poses of an image. Log Gabor has been used as feature extraction to represent the face and SVDs to reduce the redundant features. And at last, the reduced log Gabor feature is applied to the nearest neighbour classifier for recognition. The aim of this paper is to address the issues such as recognizing the individual from mug shot images, under limited number of training samples available for large degree of variations in poses. The proposed system has been evaluated for different pose variations under arbitrary tilt orientations for four different face databases.

Pose-Invariant Face Synthesis and Recognition via Sparse Coding and Symmetrical Information

Pose variation which brings illumination change, occlusion and non-linear scale variation, dramatically drops the performance of face recognition systems. In this paper, we propose a novel pose invariant face recognition method, in which we build a joint sparse coding scheme to predict face images from a certain pose to another. By introducing autoregressive regularization and symmetric information, our algorithm could achieve high robustness to local misalignment and large pose differences. Besides, we propose a new coarse pose estimation algorithm by collaborative representation classifier, which is very fast and enough accurate for our synthesis algorithm. The experiment results on multi-pose subsets of CMU-PIE and FERET database show the efficiency of the proposed method on multi-pose face recognition.

Pose invariant face recognition using biological inspired features based on ensemble of classifiers

This paper introduces a new method for view-independent face recognition by means of features inspired by the human’s visual ventral stream and ensemble of classifiers. Several sets of scale and translation invariant features are first extracted. The feature vectors are then given to a new method of ensemble of classifiers. Diversity is a crucial condition for obtaining accurate ensembles. Diversity in our method is obtained by using bootstrapped replicas of the training vectors. Different training vectors are randomly drawn from the training vectors and are connected together to form diverse training sets. Experiments were performed to validate the method on the CMU-PIE and FERET face databases. Comparison with some of the most related methods indicates that the proposed method yields better recognition rate in view independent face recognition.

Learning embedded lines of attraction by self organization for pose and expression invariant face recognition

Algorithms that mimic the computation and learning capabilities of the human brain are feasible solutions to many information-processing problems. We present a theoretical model based on the observation that images of similar visual perceptions reside in a complex manifold in an image space. To model the pattern manifold, we present a novel learning algorithm using a recurrent neural network based on the behavior of the brain. In designing a recurrent neural network, convergence dynamics of the network needs special consideration. We propose to modify this picture: If the brain remembers by converging to the state representing familiar patterns, it should also diverge from such states when presented with an unknown encoded representation of a visual image belonging to a different category. Based on this, we have developed a self-organizing line attractor to learn new patterns. A nonlinear dimensionality reduction technique is used to embed the points to a lower dimensional space that preserves the intrinsic dimensionality and metric structure of the data to enable fast and accurate recognition. Experiments performed on UMIST, CMU AMP, FRGC version-2, Japanese female face expression, and Essex Grimace databases show the effectiveness of the proposed approach in accurate recognition of complex patterns.

Robust pose invariant face recognition using coupled latent space discriminant analysis

We propose a novel pose-invariant face recognition approach which we call Discriminant Multiple Coupled Latent Subspace framework. It finds the sets of projection directions for different poses such that the projected images of the same subject in different poses are maximally correlated in the latent space. Discriminant analysis with artificially simulated pose errors in the latent space makes it robust to small pose errors caused due to a subject’s incorrect pose estimation. We do a comparative analysis of three popular latent space learning approaches: Partial Least Squares (PLSs), Bilinear Model (BLM) and Canonical Correlational Analysis (CCA) in the proposed coupled latent subspace framework. We experimentally demonstrate that using more than two poses simultaneously with CCA results in better performance. We report state-of-the-art results for pose-invariant face recognition on CMU PIE and FERET and comparable results on MultiPIE when using only four fiducial points for alignment and intensity features.

Heterogeneous Specular and Diffuse 3-D Surface Approximation for Face Recognition Across Pose

This paper proposes a novel heterogeneous specular and diffuse (HSD) 3-D surface approximation which considers spatial variability of specular and diffuse reflections in face modelling and recognition. Traditional 3-D face modelling and recognition methods constrain human faces with either the Lambertian assumption or the homogeneity assumption, resulting in suboptimal shape and texture models. The proposed HSD approach allows both specular and diffuse reflectance coefficients to vary spatially to better accommodate surface properties of real human faces. From a small number of face images of a person under different lighting conditions, 3-D shape and surface reflectivity property are estimated using a localized stochastic optimization method. The resultant personalized 3-D face model is used to render novel gallery views under different poses for recognition across pose. The proposed approach is evaluated on both synthetic and real face datasets and benchmarked against the state-of-the-art approaches. Experimental results demonstrated that it can achieve a higher level of performances in modelling accuracy, algorithm reliability, and recognition accuracy, which suggests that face modelling and recognition beyond the Lambertian and homogeneity assumptions is a feasible and better solution towards pose-invariant face recognition.

Multi-pose lipreading and audio-visual speech recognition

In this article, we study the adaptation of visual and audio-visual speech recognition systems to non-ideal visual conditions. We focus on overcoming the effects of a changing pose of the speaker, a problem encountered in natural situations where the speaker moves freely and does not keep a frontal pose with relation to the camera. To handle these situations, we introduce a pose normalization block in a standard system and generate virtual frontal views from non-frontal images. The proposed method is inspired by pose-invariant face recognition and relies on linear regression to find an approximate mapping between images from different poses. We integrate the proposed pose normalization block at different stages of the speech recognition system and quantify the loss of performance related to pose changes and pose normalization techniques. In audio-visual experiments we also analyze the integration of the audio and visual streams. We show that an audio-visual system should account for non-frontal poses and normalization techniques in terms of the weight assigned to the visual stream in the classifier.

Multi-pose lipreading and audio-visual speech recognition

In this article, we study the adaptation of visual and audio-visual speech recognition systems to non-ideal visual conditions. We focus on overcoming the effects of a changing pose of the speaker, a problem encountered in natural situations where the speaker moves freely and does not keep a frontal pose with relation to the camera. To handle these situations, we introduce a pose normalization block in a standard system and generate virtual frontal views from non-frontal images. The proposed method is inspired by pose-invariant face recognition and relies on linear regression to find an approximate mapping between images from different poses. We integrate the proposed pose normalization block at different stages of the speech recognition system and quantify the loss of performance related to pose changes and pose normalization techniques. In audio-visual experiments we also analyze the integration of the audio and visual streams. We show that an audio-visual system should account for non-frontal poses and normalization techniques in terms of the weight assigned to the visual stream in the classifier.

Pose Invariant Face Recognition Using Neuro-Biologically Inspired Features

A face recognition algorithm using biologically inspired C2 standard model features is proposed. The C2 feature extraction system, which is based on the standard model of the ventral stream of visual cortex, is modified for the extraction of facial features. The S2 units in the learning stage are tuned to frontal and profile views of the faces to provide pose invariant recognition. A new learning function is utilized to extract discriminative facial features. The spatial invariance is limited both in the local and global pooling stages. The reduced spatial invariance in the local pooling preserves the minute shape and texture details of the facial components, whereas that in the global pooling stage preserves the component location information. The faces are classified using a support vector machines classifier with linear kernel. The proposed algorithm is tested using two face datasets (MIT-CBCL, FEI) with wide variations in profiles, and compared with existing algorithms (the original C2 feature extraction algorithm and MPCALDA algorithm). The experimental results provide evidence of the better performance of the proposed approach for pose invariant face recognition, by at least 6.5%.

Unconstrained Pose-Invariant Face Recognition Using 3D Generic Elastic Models

Classical face recognition techniques have been successful at operating under well-controlled conditions; however, they have difficulty in robustly performing recognition in uncontrolled real-world scenarios where variations in pose, illumination, and expression are encountered. In this paper, we propose a new method for real-world unconstrained pose-invariant face recognition. We first construct a 3D model for each subject in our database using only a single 2D image by applying the 3D Generic Elastic Model (3D GEM) approach. These 3D models comprise an intermediate gallery database from which novel 2D pose views are synthesized for matching. Before matching, an initial estimate of the pose of the test query is obtained using a linear regression approach based on automatic facial landmark annotation. Each 3D model is subsequently rendered at different poses within a limited search space about the estimated pose, and the resulting images are matched against the test query. Finally, we compute the distances between the synthesized images and test query by using a simple normalized correlation matcher to show the effectiveness of our pose synthesis method to real-world data. We present convincing results on challenging data sets and video sequences demonstrating high recognition accuracy under controlled as well as unseen, uncontrolled real-world scenarios using a fast implementation.

Energy Normalization for Pose-Invariant Face Recognition Based on MRF Model Image Matching

A pose-invariant face recognition system based on an image matching method formulated on MRFs is presented. The method uses the energy of the established match between a pair of images as a measure of goodness-of-match. The method can tolerate moderate global spatial transformations between the gallery and the test images and alleviate the need for geometric preprocessing of facial images by encapsulating a registration step as part of the system. It requires no training on non-frontal face images. A number of innovations, such as a dynamic block size and block shape adaptation, as well as label pruning and error pre-whitening measures have been introduced to increase the effectiveness of the approach. The experimental evaluation of the method is performed on two publicly available databases. First, the method is tested on the rotation shots of the XM2VTS data set in a verification scenario. Next, the evaluation is conducted in an identification scenario on the CMU-PIE database. The method compares favorably with the existing 2D or 3D generative model-based methods on both databases in both identification and verification scenarios.

Pose-invariant face recognition by matching on multi-resolution MRFs linked by supercoupling transform

Recognition of faces in arbitrary pose is addressed in this paper. For this task, an MRF-based classification approach is proposed which employs the energy of the established match between a pair of images as a criterion of goodness-of-match. By incorporating an image matching method as part of the recognition process, the system is made robust to moderate global spatial transformations. The approach draws on a method [1] which has the potential to cope with pose changes but a direct application of which suffers from several shortcomings. In order to overcome these problems, a number of enhancements are proposed. First, by adopting a multi-scale relaxation scheme based on super coupling transform, the inference using sequential tree re-weighted message passing approach [2] is accelerated. Next, by taking advantage of a statistical shape prior for the matching, the results are regularized and constrained, making the system robust to spurious structures and outliers. For classification, both textural and structural similarities of the facial images are taken into account. The method is evaluated on two databases and promising results are obtained.

Rotation invariant complex Zernike moments features and their applications to human face and character recognition

The magnitude of Zernike moments (ZMs) has been used as rotation invariant features for classification problems in the past. Their individual real and imaginary components and phase coefficients are ignored, because they change with rotation. This study presents a new method to modify the individual real and imaginary components of ZMs which change due to image rotation. The modified real and imaginary components are then used as invariant image descriptors. The performance of the proposed method and magnitude-based ZM method is analysed on grayscale face images and binary character images in application to the fields of face recognition and character recognition, respectively. Experimental results show that the proposed method is robust to image rotation. For classification, the authors use L1-norm as the similarity measure. It is shown that the proposed method gives better recognition rate over the magnitude-based ZM method, comparatively at low orders of moment and thus it is recommended for pose invariant face recognition and also for rotation invariant character recognition. This has been proved by comparing the results of the proposed method with existing prominent methods of feature extraction in face and character recognition. On ORL database, the proposed method achieves the highest recognition rate of 96.5%, whereas a recognition rate of 99.7% is obtained on binary Roman character images.

Real-Time Pose-Invariant Face Recognition Using the Efficient Second-Order Minimization and the Pose Transforming Matrix

We propose a real-time pose-invariant face recognition algorithm from a gallery of frontal images only. (i) We modified the second-order minimization method for the active appearance model (AAM). This allows the AAM to have the ability of correct convergence with little loss of frame rate. (ii) We proposed a pose transforming matrix that can eliminate warping artifacts of the warped face image from AAM fitting. This makes it possible to train a neural network as the face recognizer with one frontal face image of each person in the gallery set. (iii) We propose a simple method for pose recognition by using neural networks to select the proper pose transforming matrix. The proposed algorithm was evaluated on a set of 2000 facial images of 10 people (200 images for each person obtained at various poses), achieving a great improvement in recognition.

Pose invariant face recognition based on hybrid-global linear regression

The paper presents a simple but efficient novel H-eigenface (Hybrid-eigenface) method for pose invariant face recognition ranging from frontal to profile view. H-eigenfaces are entirely new basis for face image representation under different poses and are used for virtual frontal view synthesis. The proposed method is based on the fact that face samples of same person under different poses are similar in terms of the combination pattern of facial features. H-eigenfaces exploit this fact and thus two H-eigenfaces under different poses capture same features of the face. Thereby providing a compact view-based subspace, which can be further used to generate virtual frontal view from inputted non-frontal face image using least square projection technique. The use of proposed methodology on FERET and ORL face database shows an impressive improvement in recognition accuracy and a distinct reduction in online computation when compared to global linear regression method.

Large-scale pose-invariant face recognition using cellular simultaneous recurrent network

In this work, we propose a novel technique for face recognition with +/-90 degrees pose variations in image sequences using a cellular simultaneous recurrent network (CSRN). We formulate the recognition problem with such large-pose variations as an implicit temporal prediction task for CSRN. We exploit a face extraction algorithm based on the scale-space method and facial structural knowledge as a preprocessing step. Further, to reduce computational cost, we obtain eigenfaces for a set of image sequences for each person and use these reduced pattern vectors as the input to CSRN. CSRN learns how to associate each face class/person in the training phase. A modified distance metric between successive frames of test and training output pattern vectors indicate either a match or mismatch between the two corresponding face classes. We extensively evaluate our CSRN-based face recognition technique using the publicly available VidTIMIT Audio-Video face dataset. Our simulation shows that for this dataset with large-scale pose variations, we can obtain an overall 77% face recognition rate.

Aligned texture map creation for pose invariant face recognition

In last years, Face recognition based on 3D techniques is an emergent technology which has demonstrated better results than conventional 2D approaches. Using texture (180° multi-view image) and depth maps is supposed to increase the robustness towards the two main challenges in Face Recognition: Pose and illumination. Nevertheless, 3D data should be acquired under highly controlled conditions and in most cases depends on the collaboration of the subject to be recognized. Thus, in applications such as surveillance or control access points, this kind of 3D data may not be available during the recognition process. This leads to a new paradigm using some mixed 2D-3D face recognition systems where 3D data is used in the training but either 2D or 3D information can be used in the recognition depending on the scenario. Following this concept, where only part of the information (partial concept) is used in the recognition, a novel method is presented in this work. This has been called Partial Principal Component Analysis (P2CA) since they fuse the Partial concept with the fundamentals of the well known PCA algorithm. This strategy has been proven to be very robust in pose variation scenarios showing that the 3D training process retains all the spatial information of the face while the 2D picture effectively recovers the face information from the available data. Furthermore, in this work, a novel approach for the automatic creation of 180° aligned cylindrical projected face images using nine different views is presented. These face images are created by using a cylindrical approximation for the real object surface. The alignment is done by applying first a global 2D affine transformation of the image, and afterward a local transformation of the desired face features using a triangle mesh. This local alignment allows a closer look to the feature properties and not the differences. Finally, these aligned face images are used for training a pose invariant face recognition approach (P2CA).