CMU PIE Research Articles

Cascaded maximum median-margin discriminant projection with its application to face recognition

This paper introduces a novel method called cascaded maximum median-margin discriminant projection (CMMDP) for robust face recognition. Initially, we introduce the maximum median-margin discriminant projection (MMDP) method, which aims to maximize the distances between class medians and their inter-class neighbors, while simultaneously minimizing the distances between class medians and their intra-class samples. To extract features at multiple levels, we extend the MMDP method to its deep version, known as CMMDP. The CMMDP algorithm incorporates the MMDP technique to learn multistage filter banks specifically designed for facial images. Afterward, we utilize straightforward binary hashing techniques for efficient indexing, along with block histograms for effective feature pooling. The CMMDP model is simple and easy to train, while taking full account of the global and local structure of the samples by maximizing the distances between samples that are dissimilar and minimizing the distances between samples that are similar after projection. This results in a low-dimensional data representation with better discriminative performance. We evaluate the performance of CMMDP on face datasets including AR, ORL, ExtYaleB and CMU PIE databases. The results indicate that CMMDP achieves superior performance than most advanced methods.

Two phase representation discriminant projection for face recognition

This paper proposes a novel approach called two-phase representation discriminant projection (TPRDP) for facial recognition. It uses two-phase representation learning to obtain a robust representation for each training sample. In the first phase, the method identifies a set of closely related samples to the training sample based on representation-based distances. In the second phase, TPRDP uses these samples to linearly encode the training sample, effectively enhancing its representation capability. After that, TPRDP constructs two graphs to capture relationships between samples. By leveraging these graphs, TPRDP seeks an optimized transformation matrix that enhances both within-class compactness and between-class separability. Through the projection of samples into a subspace using this matrix, TPRDP effectively improves the discrimination between different classes while preserving compactness within each class. Comprehensive experiments on various face datasets, including AR, CMU PIE, and ExtYaleB, confirm the effectiveness of our method.

Scatter matrix decomposition for jointly sparse learning

Orthogonal Linear Discriminant Analysis (OLDA) based on generalized Eigen-equation is widely used in the field of computer vision and pattern recognition. However, the performance of OLDA for feature extraction and classification needs to be improved as it lacks sparsity for better interpretation of the features. Moreover, computing the orthogonal sparse projections based on LDA is very difficult and is still unsolved. To solve these problems, in this paper, we propose a method called Jointly Sparse Orthogonal Linear Discriminant Analysis (JSOLDA). Different from the existing OLDA, JSOLDA is proposed from a novel viewpoint of scatter matrix decomposition. Theoretical analysis shows that OLDA can be derived by the constrained scatter matrix decomposition. In addition, by imposing L2,1-norm on the penalty term, the proposed JSOLDA can obtain the jointly sparse orthogonal projections to perform feature extraction. We also design an iterative algorithm to obtain the optimal solution. Systematic theoretical analysis between the OLDA and JSOLDA are uncovered. Both of convergence and computational complexity are also discussed. Experimental results on four data sets (i.e., COIL100, USPS, ICADAR2003 and CMU PIE) indicate that JSOLDA outperforms several well-known LDA-based and L2,1-norm based methods.

Dual collaborative representation based discriminant projection for face recognition

Collaborative representation based techniques have shown promising results for face recognition; however, most of them code the samples by taking the overall samples as a dictionary, which may contain much noise information. To tackle this problem, a new face recognition algorithm, namely dual collaborative representation based discriminant projection (DCRDP), is proposed in this paper. In DCRDP, each training sample is reconstructed via dual collaborative representation to enhance the robustness to noise information: the first collaborative representation is used to choose an appropriate dictionary with respect to the training sample, while the second collaborative representation is used to find collaborative representation relationships between samples. After dual collaborative representation, DCRDP constructs two adjacency graphs to model the similarity and dissimilarity between samples, and then finds the optimal projection matrix for dimension reduction. Experiments on ExtYaleB, AR and CMU PIE face datasets verify the superiority of DCRDP to some other state-of-the-art approaches.

A Novel Enhanced Nonnegative Matrix Factorization Method for Face Recognition

Nonnegative matrix factorization (NMF), distinguished from the approaches for holistic feature representation, is able to acquire meaningful basis images for parts-based representation. However, NMF does not utilize the data-label information and usually achieves undesirable performance in classification. To address the above-mentioned problem of NMF, this paper proposes a new enhanced NMF (ENMF) method for facial image representation and recognition. We seek to learn powerfully discriminative feature by a label-based regularizer which describes the relationship between the data. It is desired that minimizing the regularizer makes the data from the same class have high similarity and the data from different classes have low similarity. This good property will contribute to improving the performance of NMF. Therefore, we propose an objective function of ENMF by incorporating the label-based regularizer into the loss function. Subsequently, we find the stationary point of the constructed auxiliary function by means of Cardano’s formula and derive the update rules of our ENMF algorithm. The convergence of the proposed ENMF is both theoretically sound and empirically validated. Finally, the proposed ENMF method is successfully applied to face recognition. Four publicly available face datasets, namely AR, Caltech 101, Yale, and CMU PIE, are chosen for evaluations. Compared with the state-of-the-art NMF-based algorithms, the experimental results illustrate that the proposed ENMF algorithm achieves superior performance.

Recursive Sample Scaling Low-Rank Representation

The low-rank representation (LRR) method has recently gained enormous popularity due to its robust approach in solving the subspace segmentation problem, particularly those concerning corrupted data. In this paper, the recursive sample scaling low-rank representation (RSS-LRR) method is proposed. The advantage of RSS-LRR over traditional LRR is that a cosine scaling factor is further introduced, which imposes a penalty on each sample to minimize noise and outlier influence better. Specifically, the cosine scaling factor is a similarity measure learned to extract each sample’s relationship with the low-rank representation’s principal components in the feature space. In order words, the smaller the angle between an individual data sample and the low-rank representation’s principal components, the more likely it is that the data sample is clean. Thus, the proposed method can then effectively obtain a good low-rank representation influenced mainly by clean data. Several experiments are performed with varying levels of corruption on ORL, CMU PIE, COIL20, COIL100, and LFW in order to evaluate RSS-LRR’s effectiveness over state-of-the-art low-rank methods. The experimental results show that RSS-LRR consistently performs better than the compared methods in image clustering and classification tasks.

Cross-Domain Object Representation via Robust Low-Rank Correlation Analysis

Cross-domain data has become very popular recently since various viewpoints and different sensors tend to facilitate better data representation. In this article, we propose a novel cross-domain object representation algorithm (RLRCA) which not only explores the complexity of multiple relationships of variables by canonical correlation analysis (CCA) but also uses a low rank model to decrease the effect of noisy data. To the best of our knowledge, this is the first try to smoothly integrate CCA and a low-rank model to uncover correlated components across different domains and to suppress the effect of noisy or corrupted data. In order to improve the flexibility of the algorithm to address various cross-domain object representation problems, two instantiation methods of RLRCA are proposed from feature and sample space, respectively. In this way, a better cross-domain object representation can be achieved through effectively learning the intrinsic CCA features and taking full advantage of cross-domain object alignment information while pursuing low rank representations. Extensive experimental results on CMU PIE, Office-Caltech, Pascal VOC 2007, and NUS-WIDE-Object datasets, demonstrate that our designed models have superior performance over several state-of-the-art cross-domain low rank methods in image clustering and classification tasks with various corruption levels.

Local median based linear regression classification for biometric recognition

As an efficacy classification technique, linear regression based classification (LRC) has been popularly used for image recognition. However, since LRC neglects the local structure as well as dissimilarities between different classes, its performance may degrade in many real-world applications. To alleviate this problem, a robust classification method, namely local median based linear regression classification (LMLRC) is put forward in this paper. In the proposed model, based on a concept that the samples of one class having more similar features should be nearer and to handle errors brought by noise and variations between images, the test sample is linearly coded by its local median vectors of all classes; then the test sample is classified to the class which owns the biggest element in the coefficient vector. Extensive experiments on AR, CMU PIE, FRGC and PolyU Palmprint datasets show that the presented approach has superior performance to some other state-of-the-art classification algorithms.

Joint feature extraction and classification in a unified framework for cost-sensitive face recognition

Cost-sensitive face recognition is a challenging problem in pattern recognition. Due to the high-dimensional face features, cost-sensitive face recognition usually conducts feature extraction in advance, followed by the learning of classifier in reduced subspace. However, the pre-extracted face features are kept fixed and may suboptimal for subsequent classifier learning, which will degrade the final face recognition performance. Besides, most of face learners are cost insensitive. Even the cost-sensitive methods proposed for face recognition, they only incorporate the cost information in feature extraction or classification phase as an alternative. There is no doubt that some cost-sensitive information will be lost in their cost insensitive steps. To deal with these issues, this paper proposes to incorporate feature extraction and classification in a unified cost-sensitive framework for face recognition. The experimental results on three public face benchmarks, including Extended Yale B, CMU PIE and LFW datasets, demonstrate that the proposed method can significantly reduce the overall misclassification loss of face recognition system as well as the classification errors associated with high costs, when comparing with eleven state-of-the-art face learners and nine cost-sensitive methods.

Masquerade attack on biometric hashing via BiohashGAN

Masquerade attack on biometric hashing, which reconstructs the original biometric image from the given hashcode, has been given much attention recently. It is mainly used to validate the security of biometric recognition system or expand existing biometric databases like face or iris. However, an existing state-of-the-art method tends to ignore the perceptual quality of synthesized biometric images in the attack, and consequently, the synthetic images can be easily differentiated from real images. To obtain the high-perceptual-quality image which can simultaneously pass the validation of recognition system, we introduce a new target combining semantic invariability in hashing space and perceptual similarity in biometric space. In order to simulate the mapping from images to hashcodes and tackle the derivative problem related to discrete hashcodes in hashing space, we propose a DNN-based network named SimHashNet. Then we incorporate the SimHashNet into a generative adversarial network as our model named BiohashGAN to generate synthetic images form hashcodes. Experiment result on dataset CASIA-IrisV4.0-Interval and CMU PIE demonstrates that the synthetic images obtained from our model can pass the validation of recognition system and simultaneously maintain high perceptual quality.

Probabilistic Weighted induced Multi-Class Support Vector Machines for Face Recognition

This paper deals with a probabilistic weighted multi-class support vector machines for face recognition. The support vector machines (SVM) has been applied to many application fields such as pattern recognition in last decade. The support vector machines determine the hyperplane which separates largest fraction of samples of the similar class on the same side. The SVM also maximizes the distance from the either class to the separating hyperplane. It has been observed that in many realistic applications, the achieved training data is frequently tainted by outliers and noises. Support vector machines are very sensitive to outliers and noises. It may happen that a number of points in the training dataset are misplaced from their true position or even on the wrong side of the feature space. The weighted support vector machines are designed to overcome the outlier sensitivity problem of the support vector machines. The main issue in the training of the weighted support vector machines algorithm is to build up a consistent weighting model which can imitate true noise distribution in the training dataset, i.e., reliable data points should have higher weights, and the outliers should have lower weights. Therefore, the weighted support vector machines are trained depending on the weights of the data points in the training set. In the proposed probabilistic weighted multi-class support vector machines the weights are generated by probabilistic method. The weighted multi-class support vector machines have been constructed using a combination of the weighted binary support vector machines and one-against-all decision strategies. Numerous experiments have been performed on the AR, CMU PIE and FERET face databases using different experimental strategies. The experimental results show that the performance of the probabilistic weighted multi-class support vector machines is superior to the multi-class support vector machines in terms of recognition rate.

Fast single sample face recognition based on sparse representation classification

The extended sparse representation classification (ESRC) is one of the benchmark classification algorithms in the field of single sample face recognition (SSFR). However, when there are many single training samples, the execution time of ESRC cannot be acceptable in real face recognition systems. We assume the similarity principle of sparse representation under valid SSFR as that, if the test image is more similar to certain single training sample, the corresponding sparse coefficient of this single training sample may be larger, and the representation residual of this single training sample may be smaller. Based on this assumption, we propose the fast ESRC method to tackle many single training samples problem. Firstly, we propose the positive sparse coefficient based ESRC (PESRC) that selects to compute representation residuals of single training samples whose sparse coefficients of ESRC are positive. Then, we propose the statistical analysis of the sparse coefficient ratio, which is used to develop the large positive sparse coefficient based ESRC (LESRC) that calculates representation residuals of the single training samples corresponding to large positive sparse coefficients of PESRC. Finally, the experiment results on Extended Yale B, AR, CMU PIE and VGGFace2 face databases indicate that the proposed PESRC and LESRC can significantly improve the computation efficiency of ESRC. On our platform, the execution time of recognizing only one test image for ESRC or VGG + ESRC is over 130 s (the execution time of ESRC and VGG + ESRC are 562.71 s and 135.02 s) under 9125 single training samples, whereas the execution times of PESRC, LESRC, VGG + PESRC, and VGG + LESRC are 2.20s, 0.23 s, 0.71 s, and 0.03 s respectively.

A Novel Approach of Face Recognition Using Optimized Adaptive Illumination–Normalization and KELM

Light variations from different directions on the face images cause severe performance degradation in face recognition system. These variations should be nullified or suppressed so that the recognition performance can be improved. Here, the objective is to develop a robust and pragmatic method which can compensate the effect of uncontrolled light incident from different directions on the person’s face. To normalize the effect of unpredictable illumination variations from face images, a novel illumination–normalization method is proposed by utilizing the fractional discrete cosine transform (Fr-DCT) and a nonlinear modifier. Fr-DCT provides a generalized α-domain in which illumination variation’s effect can be better processed. The proposed method adaptively selects and processes the low-frequency α-domain coefficients depending upon the light variations incident on the face image. The illumination-normalized images are classified with the help of a non-iterative classifier. The parameters of the proposed method are optimized with the help of genetic algorithm. The proposed method is tested over Extended YALE B, AR, CMU PIE and YALE face databases. Very promising results have been achieved with percentage error rate zero for CMU PIE and subset 3 of Extended YALE B, 0.38% for subset 4 and 0.55% for subset 5. On AR database, error rate is 0.5% when only illumination-varying images are considered for testing. The comparison of the results of the proposed method with those of the existing state-of-the-art methods clearly establishes the superiority of the proposed method.

Illumination-robust face recognition based on deep convolutional neural networks architectures

<p><span>In the last decade, facial recognition techniques are considered the most important fields of research in biometric technology. In this research paper, we present a Face Recognition (FR) system divided into three steps: The Viola-Jones face detection algorithm, facial image enhancement using Modified Contrast Limited Adaptive Histogram Equalization algorithm (M-CLAHE), and feature learning for classification. For learning the features followed by classification we used VGG16, ResNet50 and Inception-v3 Convolutional Neural Networks (CNN) architectures for the proposed system. Our experimental work was performed on the Extended Yale B database and CMU PIE face database. Finally, the comparison with the other methods on both databases shows the robustness and effectiveness of the proposed approach. Where the Inception-v3 architecture has achieved a rate of 99, 44% and 99, 89% respectively.</span></p>

Template Based Pose and Illumination Invariant Face Recognition

This article presents a method “Template based pose and illumination invariant face recognition”. We know that pose and Illumination are important variants where we cannot find proper face images for a given query image. As per the literature, previous methods are also not accurately calculating the pose and Illumination variants of a person face image. So we concentrated on pose and Illumination. Our System firstly calculates the face inclination or the pose of the head of a person with various mathematical methods. Then Our System removes the Illumination from the image using a Gabor phase based illumination invariant extraction strategy. In this strategy, the system normalizes changing light on face images, which can decrease the impact of fluctuating Illumination somewhat. Furthermore, a lot of 2D genuine Gabor wavelet with various orientations is utilized for image change, and numerous Gabor coefficients are consolidated into one entire in thinking about spectrum and phase. Finally, the light invariant is acquired by separating the phase feature from the consolidated coefficients. Then after that, the obtained Pose and illumination invariant images are convolved with Gabor filters to obtain Gabor images. Then templates will be extracted from these Gabor images and one template average is generated. Then similarity measure will be performed between query image template average and database images template averages. Finally the most similar images will be displayed to the user. Exploratory results on PubFig database, Yale B and CMU PIE face databases show that our technique got a critical improvement over other related strategies for face recognition under enormous pose and light variation conditions.

A novel non-linear modifier for adaptive illumination normalization for robust face recognition

In this paper, a novel approach is presented for adaptive illumination normalization for face recognition under varying illuminations due to change in angle of light projection. Illumination normalization is performed over some of the low frequency discrete Cosine transform (DCT) coefficients which are computed adaptively based upon the significance of alteration of these coefficients. These are, then, modified using a non-linear modifier. The significance of the proposed approach is that the approach is adaptive in normalizing the illumination from the face images as the number of low frequency DCT coefficients that need to be modified using non-linear modifier are selected on the basis of variations of illumination present in the image. This variation in the illumination is also used in developing the non-linear modifier. The proposed approach is important in such a way that the level of illumination variations decides the computations needed i.e. lesser in comparison to the other existing state-of-art approaches as large number of frequency coefficients are not altered. The proposed approach is tested over various face databases: YALE B, Extended YALE B, CMU PIE, AR and YALE face database. The experimental results clearly reveal that the performance of the proposed approach is significantly better than the existing approaches of illumination normalization for face recognition. With the proposed approach, 100% accuracy is attained on all the subsets of YALE B, CMU PIE and on Subset 3 of Extended YALE B database. Promising results are achieved over remaining face databases as well.

Generalized DCT and DWT hybridization based robust feature extraction for face recognition

Pose, expression and light variations are the major complications in recognition of face images. Hence, a robust technique is introduced in this paper which deals with variations in light in the face images. Hybridization of fractional discrete Cosine transform (FrDCT) and DWT are utilized as the feature extraction technique which helps in normalizing the light variations in the face images. DWT helps in obtaining the global behaviour of the light compensated face image. Also, use of DWT with FrDCT helps in obtaining the redundancy which is difficult to obtain with FrDCT alone. Therefore, it is obtained with DWT and after that, FrDCT is exploited for the local correlation. The proposed method is experimented and investigated on YALE, Yale B and CMU PIE face databases and very good performance outcome has been achieved as compared to the other state of art methods.

Diagonal Symmetric Pattern-Based Illumination Invariant Measure for Severe Illumination Variation Face Recognition

The center symmetric pattern (CSP) was widely used in the local binary pattern based facial feature, whereas never used to develop the illumination invariant measure in the literature. This paper proposes a novel diagonal symmetric pattern (DSP) to develop the illumination invariant measure for severe illumination variation face recognition. Firstly, the subtraction of two diagonal symmetric pixels is defined as the DSP unit in the face local region, which may be positive or negative. The DSP model is obtained by combining the positive and negative DSP units in the even $\times $ even block region. Then, the DSP model can be used to generate several DSP images based on the $2\times 2$ block or the $4\times 4$ block by controlling the proportions of positive and negative DSP units, which results in the DSP2 image or the DSP4 image. The single DSP2 or DSP4 image with the arctangent function can develop the DSP2-face or the DSP4-face. Multi DSP2 or DSP4 images employ the extended sparse representation classification (ESRC) as the classifier that can form the DSP2 images based classification (DSP2C) or the DSP4 images based classification (DSP4C). Further, the DSP model is integrated with the pre-trained deep learning (PDL) model to construct the DSP-PDL model. Finally, the experimental results on the Extended Yale B, CMU PIE, AR, and VGGFace2 face databases indicate that the proposed methods are efficient to tackle severe illumination variations.

Face Image Set Recognition Based On Improved HOG-NMF and Convolutional Neural Networks

Objective Face recognition can be affected by unfavorable factors such as illumination, posture and expression, but the face image set is a collection of people’s various angles, different illuminations and even different expressions, which can effectively reduce these adverse effects and get higher face recognition rate. In order to make the face image set have higher recognition rate, a new method of combining face image set recognition is proposed, which combines an improved Histogram of Oriented Gradient (HOG) feature and Convolutional Neural Network (CNN). Method The method firstly segments the face images to be identified and performs HOG to extract features of the segmented images. Secondly, calculate the information entropy contained in each block as a weight coefficient of each block to form a new HOG features, and non-negative matrix factorization (NMF) is applied to reduce HOG features. Then the reduced-dimensional HOG features are modeled as image sets which keep your face details as much as possible. Finally, the modeled image sets are classified by using a convolutional neural network. Result The experimental results show that compared with the simple CNN method and the HOG-CNN method, the recognition rate of the method on the CMU PIE face set is increased by about 4%~10%. Conclusion The method proposed in this paper has more details of the face, overcomes the adverse effects, and improves the accuracy.

Cropped and Extended Patch Collaborative Representation Face Recognition for a Single Sample Per Person

Face recognition for a single sample per person (SSPP) is a challenging task due to the lack of sufficient sample information. In this paper, in order to raise the performance of face recognition for SSPP, we propose an algorithm of cropped and extended patch collaborative representation for a single sample per person (CEPCRC). Considering the fact that patch-based method can availably avoid the effect of variations, and the fact that intra-class variations learned from a generic training set can sparsely represent the possible facial variations, thus, we extend patch collaborative representation based classification into the SSPP scenarios by using the intra-class variant dictionary and learn patch weight by exploiting regularized margin distribution optimization. For more complementary information, we construct multiple training samples by the means of cropping. Experimental results in the CMU PIE, Extended Yale B, AR, and LFW datasets demonstrate that CEPCRC performs better compared to the related algorithms.