CMU PIE Database Research Articles

Face Recognition Using Spatially Constrained Earth Mover's Distance

Face recognition is a challenging problem, especially when the face images are not strictly aligned (e.g., images can be captured from different viewpoints or the faces may not be accurately cropped by a human or automatic algorithm). In this correspondence, we investigate face recognition under the scenarios with potential spatial misalignments. First, we formulate an asymmetric similarity measure based on Spatially constrained Earth Mover's Distance (SEMD), for which the source image is partitioned into nonoverlapping local patches while the destination image is represented as a set of overlapping local patches at different positions. Assuming that faces are already roughly aligned according to the positions of their eyes, one patch in the source image can be matched only to one of its neighboring patches in the destination image under the spatial constraint of reasonably small misalignments. Because the similarity measure as defined by SEMD is asymmetric, we propose two schemes to combine the two similarity measures computed in both directions. Moreover, we adopt a distance-as-feature approach by treating the distances to the reference images as features in a Kernel Discriminant Analysis (KDA) framework. Experiments on three benchmark face databases, namely the CMU PIE, FERET, and FRGC databases, demonstrate the effectiveness of the proposed SEMD.

A Subspace Model-Based Approach to Face Relighting Under Unknown Lighting and Poses

We present a new approach to face relighting by jointly estimating the pose, reflectance functions, and lighting from as few as one image of a face. Upon such estimation, we can synthesize the face image under any prescribed new lighting condition. In contrast to commonly used face shape models or shape-dependent models, we neither recover nor assume the 3-D face shape during the estimation process. Instead, we train a pose- and pixel-dependent subspace model of the reflectance function using a face database that contains samples of pose and illumination for a large number of individuals (e.g., the CMU PIE database and the Yale database). Using this subspace model, we can estimate the pose, the reflectance functions, and the lighting condition of any given face image. Our approach lends itself to practical applications thanks to many desirable properties, including the preservation of the non-Lambertian skin reflectance properties and facial hair, as well as reproduction of various shadows on the face. Extensive experiments show that, compared to recent representative face relighting techniques, our method successfully produces better results, in terms of subjective and objective quality, without reconstructing a 3-D shape.

3D Head Orientation Estimation and Expression Influence Minimization using Characteristic Points of Face

The method of minimization face expression influence for evaluation of head orientation in space is proposed and investigated. In our previous works 18 characteristic face points were selected for the head orientation evaluation. It is difficult to track all 18 points in real time. It was found by way of mathematical simulation that number of points 7 is optimal number for head orientation tracking. The points were selected at the face places which obtain minimal changes because of face expression. Mathematical simulation helped to establish that errors for head rotations could reach value 13 degrees (for surprised expression), and mean error is about 6-8 degrees. After correction of face specific points coordinates by mean value of change caused by face expression the errors were reduced until range 0.1-0.3 degrees. Intervals of possible errors for every expression and some head orientation angles were calculated. The method was tested using CMU PIE database of face images. The obtained interval for head orientation angles errors is 1.5-2.8. The correspondence of 78-90% was obtained between results of experimental errors and results of mathematical simulation. Ill. 6, bibl. 7 (in English; summaries in English, Russian and Lithuanian).

SYMBOLIC FACTORIAL DISCRIMINANT ANALYSIS FOR ILLUMINATION INVARIANT FACE RECOGNITION

In this paper, a new appearance-based technique called symbolic factorial discriminant analysis (symbolic FDA) is explored for face representation and recognition under varying illumination conditions. In the past few years, many appearance-based methods have been proposed to model image variations of human faces under different lighting conditions using single valued variables to represent the facial features. In the proposed symbolic factorial discriminant analysis method, we extract interval type discriminating features, which are robust to illumination changes. The minimum distance classifier with symbolic dissimilarity measure is used for classification. The proposed method has been successfully tested for face recognition using three databases, namely, Yale Face database B, CMU PIE database and Harvard database. The experimental results have demonstrated the effective performance of this method.

Face Recognition Based on Nonlinear Feature Approach

Feature extraction techniques are widely used to reduce the complexity high dimensional data. Nonlinear feature extraction via Locally Linear Embedding (LLE) has attracted much attention due to their high performance. In this paper, we proposed a novel approach for face recognition to address the challenging task of recognition using integration of nonlinear dimensional reduction Locally Linear Embedding integrated with Local Fisher Discriminant Analysis (LFDA) to improve the discriminating power of the extracted features by maximize between-class while within-class local structure is preserved. Extensive experimentation performed on the CMU-PIE database indicates that the proposed methodology outperforms Benchmark methods such as Principal Component Analysis (PCA), Fisher Discrimination Analysis (FDA). The results showed that 95% of recognition rate could be obtained using our proposed method.

Retinex method based on adaptive smoothing for illumination invariant face recognition

In this paper, we propose the Retinex method for illumination invariant face recognition developed on the basis of adaptive smoothing technology. By the well-known Retinex theory, illumination is generally estimated and normalized by smoothing the input image first and then dividing the estimate into the original input image. Therefore, performance mainly depends on how good the estimated illumination is. The proposed method estimates illumination by iteratively convolving the input image with a 3 × 3 smoothing mask weighted by a coefficient via combining two measures of the illumination discontinuity at each pixel. We address a couple of additional concepts, which are designed to be suitable especially for face images. One is the new conduction function for adaptive smoothing, and the other is the smoothing constraint for more accurate description of real environments. In this way, we can achieve an efficient illumination normalization in which face images with even strong shadows are normalized efficiently. The proposed method is evaluated based on Yale face database B, CMU PIE database and AR face database by applying PCA. The comparative results indicate that the proposed method present consistent and promising results even when images under harsh illumination conditions are used as a training set.

Toward Pose-Invariant 2-D Face Recognition Through Point Distribution Models and Facial Symmetry

This paper proposes novel ways to deal with pose variations in a 2-D face recognition scenario. Using a training set of sparse face meshes, we built a point distribution model and identified the parameters which are responsible for controlling the apparent changes in shape due to turning and nodding the head, namely the pose parameters. Based on them, we propose two approaches for pose correction: 1) a method in which the pose parameters from both meshes are set to typical values of frontal faces, and 2) a method in which one mesh adopts the pose parameters of the other one. Finally, we obtain pose corrected meshes and, taking advantage of facial symmetry, virtual views are synthesized via Thin Plate Splines-based warping. Given that the corrected images are not embedded into a constant reference frame, holistic methods are not suitable for feature extraction. Instead, the virtual faces are fed into a system that makes use of Gabor filtering for recognition. Unlike other approaches that warp faces onto a mean shape, we show that if only pose parameters are modified, client specific information remains in the warped image and discrimination between subjects is more reliable. Statistical analysis of the authentication results obtained on the XM2VTS database confirm the hypothesis. Also, the CMU PIE database is used to assess the performance of the proposed methods in an identification scenario where large pose variations are present, achieving state-of-the-art results and outperforming both research and commercial techniques.

Choosing Parameters of Kernel Subspace LDA for Recognition of Face Images Under Pose and Illumination Variations

This paper addresses the problem of automatically tuning multiple kernel parameters for the kernel-based linear discriminant analysis (LDA) method. The kernel approach has been proposed to solve face recognition problems under complex distribution by mapping the input space to a high-dimensional feature space. Some recognition algorithms such as the kernel principal components analysis, kernel Fisher discriminant, generalized discriminant analysis, and kernel direct LDA have been developed in the last five years. The experimental results show that the kernel-based method is a good and feasible approach to tackle the pose and illumination variations. One of the crucial factors in the kernel approach is the selection of kernel parameters, which highly affects the generalization capability and stability of the kernel-based learning methods. In view of this, we propose an eigenvalue-stability-bounded margin maximization (ESBMM) algorithm to automatically tune the multiple parameters of the Gaussian radial basis function kernel for the kernel subspace LDA (KSLDA) method, which is developed based on our previously developed subspace LDA method. The ESBMM algorithm improves the generalization capability of the kernel-based LDA method by maximizing the margin maximization criterion while maintaining the eigenvalue stability of the kernel-based LDA method. An in-depth investigation on the generalization performance on pose and illumination dimensions is performed using the YaleB and CMU PIE databases. The FERET database is also used for benchmark evaluation. Compared with the existing PCA-based and LDA-based methods, our proposed KSLDA method, with the ESBMM kernel parameter estimation algorithm, gives superior performance.

Locally Linear Regression for Pose-Invariant Face Recognition

The variation of facial appearance due to the viewpoint (/pose) degrades face recognition systems considerably, which is one of the bottlenecks in face recognition. One of the possible solutions is generating virtual frontal view from any given nonfrontal view to obtain a virtual gallery/probe face. Following this idea, this paper proposes a simple, but efficient, novel locally linear regression (LLR) method, which generates the virtual frontal view from a given nonfrontal face image. We first justify the basic assumption of the paper that there exists an approximate linear mapping between a nonfrontal face image and its frontal counterpart. Then, by formulating the estimation of the linear mapping as a prediction problem, we present the regression-based solution, i.e., globally linear regression. To improve the prediction accuracy in the case of coarse alignment, LLR is further proposed. In LLR, we first perform dense sampling in the nonfrontal face image to obtain many overlapped local patches. Then, the linear regression technique is applied to each small patch for the prediction of its virtual frontal patch. Through the combination of all these patches, the virtual frontal view is generated. The experimental results on the CMU PIE database show distinct advantage of the proposed method over Eigen light-field method.

Nonlinear Discriminant Analysis on Embedded Manifold

Traditional manifold learning algorithms, such as ISOMAP, LLE, and Laplacian Eigenmap, mainly focus on uncovering the latent low-dimensional geometry structure of the training samples in an unsupervised manner where useful class information is ignored. Therefore, the derived low-dimensional representations are not necessarily optimal in discriminative capability. In this paper, we study the discriminant analysis problem by considering the nonlinear manifold structure of data space. To this end, firstly, a new clustering algorithm, called Intra-Cluster Balanced K-Means (ICBKM), is proposed to partition the samples into multiple clusters while ensure that there are balanced samples for the classes within each cluster; approximately, each cluster can be considered as a local patch on the embedded manifold. Then, the local discriminative projections for different clusters are simultaneously calculated by optimizing the global Fisher Criterion based on the cluster weighted data representation. Compared with traditional linear/kernel discriminant analysis (KDA) algorithms, our proposed algorithm has the following characteristics: 1) it essentially is a KDA algorithm with specific geometry-adaptive-kernel tailored to the specific data structure, in contrast to traditional KDA in which the kernel is fixed and independent to the data set; 2) it is approximately a locally linear while globally nonlinear discriminant analyzer; 3) it does not need to store the original samples for computing the low-dimensional representation of a new data; and 4) it is computationally efficient compared with traditional KDA when the sample number is large. The toy problem on artificial data demonstrates the effectiveness of our proposed algorithm in deriving discriminative representations for problems with nonlinear classification hyperplane. The face recognition experiments on YALE and CMU PIE databases show that our proposed algorithm significantly outperforms linear discriminant analysis (LDA) as well as Mixture LDA, and has higher accuracy than KDA with traditional kernels

Gabor feature-based face recognition using supervised locality preserving projection

This paper introduces a novel Gabor-based supervised locality preserving projection (GSLPP) method for face recognition. Locality preserving projection (LPP) is a recently proposed method for unsupervised linear dimensionality reduction. LPP seeks to preserve the local structure which is usually more significant than the global structure preserved by principal component analysis (PCA) and linear discriminant analysis (LDA). In this paper, we investigate its extension, called supervised locality preserving projection (SLPP), using class labels of data points to enhance its discriminant power in their mapping into a low-dimensional space. The GSLPP method, which is robust to variations of illumination and facial expression, applies the SLPP to an augmented Gabor feature vector derived from the Gabor wavelet representation of face images. We performed comparative experiments of various face recognition schemes, including the proposed GSLPP method, PCA method, LDA method, LPP method, the combination of Gabor and PCA method (GPCA) and the combination of Gabor and LDA method (GLDA). Experimental results on AR database and CMU PIE database show superior of the novel GSLPP method.

Estimating Illumination Parameters Using Spherical Harmonics Coefficients in Frequency Space

An algorithm is presented for estimating the direction and strength of point light with the strength of ambient illumination. Existing approaches evaluate these illumination parameters directly in the high dimensional image space, while we estimate the parameters in two steps: first by projecting the image to an orthogonal linear subspace based on spherical harmonic basis functions and then by calculating the parameters in the low dimensional subspace. The test results using the CMU PIE database and Yale Database B show the stability and effectiveness of the method. The resulting illumination information can be used to synthesize more realistic relighting images and to recognize objects under variable illumination.

Multilinear Discriminant Analysis for Face Recognition

There is a growing interest in subspace learning techniques for face recognition; however, the excessive dimension of the data space often brings the algorithms into the curse of dimensionality dilemma. In this paper, we present a novel approach to solve the supervised dimensionality reduction problem by encoding an image object as a general tensor of second or even higher order. First, we propose a discriminant tensor criterion, whereby multiple interrelated lower dimensional discriminative subspaces are derived for feature extraction. Then, a novel approach, called k-mode optimization, is presented to iteratively learn these subspaces by unfolding the tensor along different tensor directions. We call this algorithm multilinear discriminant analysis (MDA), which has the following characteristics: 1) multiple interrelated subspaces can collaborate to discriminate different classes, 2) for classification problems involving higher order tensors, the MDA algorithm can avoid the curse of dimensionality dilemma and alleviate the small sample size problem, and 3) the computational cost in the learning stage is reduced to a large extent owing to the reduced data dimensions in k-mode optimization. We provide extensive experiments on ORL, CMU PIE, and FERET databases by encoding face images as second- or third-order tensors to demonstrate that the proposed MDA algorithm based on higher order tensors has the potential to outperform the traditional vector-based subspace learning algorithms, especially in the cases with small sample sizes.

LINEAR AND NONLINEAR FEATURE-BASED FUSION ALGORITHMS FOR FACE RECOGNITION

Integration of various face recognition algorithms has proved to be a feasible approach to improve the performance of a face recognition system. Different face recognition algorithms are often based on different representations of the input patterns or on extracted features and hence may complement each other. Linear and nonlinear feature based algorithms can capture and handle different kinds of variations, such as pose, illumination and expression variations. To make full use of the different advantages of different classifiers, we propose combining four linear and nonlinear face recognition algorithms via a weighted combination scheme to improve the recognition performance of a face recognition system. The FERET, YaleB and CMU PIE database are used for evaluating the combination scheme and the results confirm the effectiveness of the proposed combination scheme.

Multi‐subregion‐based probabilistic approach to pose‐invariant face recognition

AbstractCurrent automatic facial recognition systems are not robust to changes in illumination, pose, facial expression, and occlusion. In this paper, in order to address the problem of pose change, we propose an algorithm based on a probabilistic approach to face recognition that takes into account the pose difference between probe and gallery images. By using a large facial image database, the CMU PIE database, which contains images of the same set of people taken from many different angles, we have developed a probabilistic model of how facial features change as the pose changes. This model enables us to make our face recognition system more robust to changes of pose in the probe image. The experimental results show that this approach achieves a better recognition rate than conventional face recognition methods over a much larger range of poses. For example, when the gallery contains only images of a frontal face and the probe image varies its pose orientation, the recognition rate shows less than a 10% difference until the probe pose begins to differ by more than 45°, whereas the recognition rate of a PCA‐based method begins to drop at differences as small as 10°, and that of a representative commercial system begins to drop at 30°. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(8): 68–76, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20437

Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain

This paper presents a novel illumination normalization approach for face recognition under varying lighting conditions. In the proposed approach, a discrete cosine transform (DCT) is employed to compensate for illumination variations in the logarithm domain. Since illumination variations mainly lie in the low-frequency band, an appropriate number of DCT coefficients are truncated to minimize variations under different lighting conditions. Experimental results on the Yale B database and CMU PIE database show that the proposed approach improves the performance significantly for the face images with large illumination variations. Moreover, the advantage of our approach is that it does not require any modeling steps and can be easily implemented in a real-time face recognition system.

TWO-STEP SINGLE PARAMETER REGULARIZATION FISHER DISCRIMINANT METHOD FOR FACE RECOGNITION

In face recognition tasks, Fisher discriminant analysis (FDA) is one of the promising methods for dimensionality reduction and discriminant feature extraction. The objective of FDA is to find an optimal projection matrix, which maximizes the between-class-distance and simultaneously minimizes within-class-distance. The main limitation of traditional FDA is the so-called Small Sample Size (3S) problem. It induces that the within-class scatter matrix is singular and then the traditional FDA fails to perform directly for pattern classification. To overcome 3S problem, this paper proposes a novel two-step single parameter regularization Fisher discriminant (2SRFD) algorithm for face recognition. The first semi-regularized step is based on a rank lifting theorem. This step adjusts both the projection directions and their corresponding weights. Our previous three-to-one parameter regularized technique is exploited in the second stage, which just changes the weights of projection directions. It is shown that the final regularized within-class scatter matrix approaches the original within-class scatter matrix as the single parameter tends to zero. Also, our method has good computational complexity. The proposed method has been tested and evaluated with three public available databases, namely ORL, CMU PIE and FERET face databases. Comparing with existing state-of-the-art FDA-based methods in solving the S3 problem, the proposed 2SRFD approach gives the best performance.

Pose and Illumination Invariant Face Recognition Based on 3D Face Reconstruction

Pose and illumination changings from picture to picture are two main barriers toward full automatic face recognition. In this paper, a novel method to handle both pose and lighting conditions simultaneously is proposed, which calibrates the pose and lighting to a predefined reference condition through an illumination invariant 3D face reconstruction. First, some located facial landmarks and a priori statistical deformable 3D model are used to recover an elaborate 3D shape. Based on the recovered 3D shape, the texture calibrated to a standard illumination is generated by spherical harmonics ratio image and finally the illumination independent 3D face is reconstructed completely. The proposed method combines the strength of statistical deformable model to describe the shape information and the compact representations of the illumination in spherical frequency space, and handles both the pose and illumination variation simultaneously. This algorithm can be used to synthesize virtual views of a given face image and enhance the performance of face recognition. Experimental results on CMU PIE database show that this method can significantly improve the accuracy of the existing face recognition method when pose and illumination are inconsistent between gallery and probe sets.

Multiresolution face recognition

In this paper the contribution of multiresolution analysis to the face recognition performance is examined. We refer to the paradigm that in classification tasks, the use of multiple observations and their judicious fusion at the data, feature or decision level improves the correct decision performance. In our proposed method, prior to the subspace projection operation like principal or independent component analysis, we employ multiresolution analysis to decompose the image into its subbands. Our aim is to search for the subbands that are insensitive to the variations in expression and in illumination. The classification performance is improved by fusing the information coming from the subbands that attain individually high correct recognition rates. The proposed algorithm is tested on face images that differ in expression or illumination separately, obtained from CMU PIE, FERET and Yale databases. Significant performance gains are attained, especially against illumination perturbations.

Feature selection in the independent component subspace for face recognition

This paper addresses the feature selection problem for face recognition in the independent component subspace. While there exists, at least, energy principle to guide the selection of the principle components, the independent components (ICs) are devoid of any energy ranking, and must therefore selected based on their discriminatory power. In addition the independent component features can be selected starting from a much larger pool, or from a combination pool of ICA and PCA features. Four feature selection schemes have been comparatively assessed, and feature subsets are tested on a face database constructed from CMU PIE and FERET databases. The discriminatory features from larger pools are observed to be concentrated around fiduciary spatial details of the nose, the eyes and the facial contour. Overall, face recognition benefits from the feature selection of ICA or PCA components and from the combination of ICA and PCA feature pools.