Low-resolution Face Images Research Articles

Explicit and implicit employment of edge-related information in super-resolving distant faces for recognition

Edges and related features play significant role in discriminating face images. But those features are not enough informative when the face images are captured from a distance (e.g., video surveillance). Traditionally, those features are enhanced by super-resolving low-resolution grayscale face images. In this paper, we demonstrate a superior performance by directly considering such features (continuous gradient value, also known as edginess) in the super-resolution process. Edginess features are extracted using 1-D processing of image. This process is carried out along different directions to obtain partial evidences, which are combined to detect the person's identity. Here, super-resolution of the face image and its recognition has been performed in sparse domain framework. The explicit usage of edginess feature in the proposed approach shows considerable improvement in both recognition performance as well as computational time, as only the patches related to strong edges are considered for super-resolution. In addition to that, the edginess feature gives improved recognition rate when it is preserved implicitly during super-resolution in grayscale domain.

From Local Geometry to Global Structure: Learning Latent Subspace for Low-resolution Face Image Recognition

In this letter, we propose a novel approach for learning coupled mappings to improve the performance of low-resolution (LR) face image recognition. The coupled mappings aim to project the LR probe images and high-resolution (HR) gallery images into a unified latent subspace, which is efficient to measure the similarity of face images with different resolutions. In the training phase, we first construct local optimization for each training sample according to the relationship of neighboring data points. The local optimization aims to: (1) ensure the consistency for each LR face image and corresponding HR one; (2) model the intrinsic geometric structure between each given sample and its neighbors; and (3) preserve the discriminative information across different subjects. We finally incorporate the local optimizations together for building the global structure. The coupled mappings can be learned by solving a standard eigen-decomposition problem, which avoids the small-sample-size problem. Experimental results demonstrate the effectiveness of the proposed method on public face databases.

Improving cross-resolution face matching using ensemble-based co-transfer learning.

Face recognition algorithms are generally trained for matching high-resolution images and they perform well for similar resolution test data. However, the performance of such systems degrades when a low-resolution face image captured in unconstrained settings, such as videos from cameras in a surveillance scenario, are matched with high-resolution gallery images. The primary challenge, here, is to extract discriminating features from limited biometric content in low-resolution images and match it to information rich high-resolution face images. The problem of cross-resolution face matching is further alleviated when there is limited labeled positive data for training face recognition algorithms. In this paper, the problem of cross-resolution face matching is addressed where low-resolution images are matched with high-resolution gallery. A co-transfer learning framework is proposed, which is a cross-pollination of transfer learning and co-training paradigms and is applied for cross-resolution face matching. The transfer learning component transfers the knowledge that is learnt while matching high-resolution face images during training to match low-resolution probe images with high-resolution gallery during testing. On the other hand, co-training component facilitates this transfer of knowledge by assigning pseudolabels to unlabeled probe instances in the target domain. Amalgamation of these two paradigms in the proposed ensemble framework enhances the performance of cross-resolution face recognition. Experiments on multiple face databases show the efficacy of the proposed algorithm and compare with some existing algorithms and a commercial system. In addition, several high profile real-world cases have been used to demonstrate the usefulness of the proposed approach in addressing the tough challenges.

Research on Very Low Resolution Face Image Reconstruction

The resolution of the face image in video may lower than 16*16 in environmental such as ultra long distance, poor illumination and so on, with the very low resolution (VLR) face image the existing face super-resolution (SR) methods do not perform well. In this paper, we proposes a new algorithms by learning the relationship between high-resolution (HR) image space and the VLR image space for face SR.A new constrain, new data constrain are design for reconstruct HR face image form VLR face image. The Experiment results show that the proposed method can recover a clear face image from the VLR face image.

Very Low Resolution Face Image Super-Resolution Based on DCT

This paper introduces the Very Low Resolution (VLR) problem in the fleld of face image super-resolution reconstruction. With the extensive application of the surveillance cameras, the very low resolution problem happens in many face recognition applications, which has brought great inconvenience to users. In order to research this problem, this paper proposes the improved DCT algorithm, and our flnal experimental results have compared with the common DCT algorithm, Cubic B-Spline interpolation algorithm and Baker et al’s method. Experimental results show that the improved DCT algorithm can get better performance in the very low resolution face image super-resolution reconstruction problem.

Structural similarity classifier for facial expression recognition

A novel Gabor filter structural similarity algorithm (GFSSIM) is proposed for facial expression recognition (FER) on noisy images. Low-resolution facial images with low SNRs are specifically dealt with FER system. The features are extracted using 40 Gabor filters, and a feature subset is selected for classification. The test image is classified based on proposed GFSSIM algorithm. The experimental results show that the recognition rate for heavily deteriorated images outperforms the conventional classifier method. In addition, the proposed method is more efficient from the computational complexity point of view.

Face hallucination with imprecise-alignment using iterative sparse representation

Existing face hallucination methods assume that the face images are well-aligned. However, in practice, given a low-resolution face image, it is very difficult to perform precise alignment. As a result, the quality of the super-resolved image is degraded dramatically. In this paper, we propose a near frontal-view face hallucination method which is robust to face image mis-alignment. Based on the discriminative nature of sparse representation, we propose a global face sparse representation model that can reconstruct images with mis-alignment variations. We further propose an iterative method combining the global sparse representation and the local linear regression using the Expectation Maximization (EM) algorithm, in which the face hallucination is converted into a parameter estimation problem with incomplete data. Since the proposed algorithm is independent of the face similarity resulting from precise alignment, the proposed algorithm is robust to mis-alignment. In addition, the proposed iterative manner not only combines the merits of the global and local face hallucination, but also provides a convenient way to integrate different strategies to handle the mis-alignment problem. Experimental results show that the proposed method achieves better performance than existing methods, especially for mis-aligned face images.

Face image super-resolution through locality-induced support regression

In this paper we propose a novel face image super-resolution (SR) method named Locality-induced Support Regression (LiSR). Given a low-resolution (LR) input patch, we learn a mapping function between the local support LR and high-resolution (HR) patch pairs to predict its HR version. The support can be obtained from the LR or HR patch manifolds, which leads to two varieties of LiSR, namely LR patch guided LiSR (LR-LiSR) and HR patch guided LiSR (HR-LiSR). LR-LiSR directly learns the mapping function between local support LR/HR patch pairs given an input LR patch. As for HR-LiSR, the support and a mapping function will be iteratively learned to update the target HR patch. The key advantages of our proposed framework are two-fold: (1) the strong regularization of “same representation” of prior work [1,2] is relaxed to the same support, and hence much flexibility can be given to the learned mapping function; (2) we define the support in the LR or HR patch manifold space by incorporating the locality constraint, which can well preserve the manifold structure of the training set. Experimental results reported on both simulated LR face images and real-world datasets demonstrate the effectiveness of the proposed method.

Super-Resolution for Surveillance Facial Images via Shape Prior and Residue Compensation

Super-resolution (SR) methods are widely used to enhance the resolution of input images such as low-resolution (LR) remote or surveillance images. The key problem of SR algorithm is to introduce a proper prior for high-resolution (HR) image reconstruction. Usually pixel feature based similarity measure is used as prior. While in real surveillance applications, images are often disturbed by noise so that it is not enough to only use the prior from pixel domain to achieve satisfied high-resolution. In this paper, a two-phase based face hallucination approach via shape prior (SP) is proposed. Firstly, the Active Appearance Model (AAM) is used as shape prior to reconstruct the global face. Then the high frequency information as detailed inartificial facial features of HR face image is compensated to the global face. Experiments show that the proposed algorithm improves the subjective and objective quality of the input LR facial images and outperform many states-of-the- art superresolution methods.

A two-step face hallucination approach for video surveillance applications

In this paper we propose a novel face hallucination algorithm to synthesize a high-resolution face image from several low-resolution input face images. Face hallucination normally uses two models: a global parametric model which synthesizes the global face shapes from eigenfaces, and a local parametric model which enhances the local high frequency details. We follow a similar process to develop a robust face hallucination algorithm. First, we obtain eigenfaces from a number of low resolution face images segmented from a video sequence using a face tracking algorithm. Then we compute the difference between an interpolated low-resolution face and a mean face, and use this difference as the query to retrieve an approximate sparse eigenface representation. The eigenfaces are combined using the coefficients obtained from the sparse representation and added into the interpolated low-resolution face. In this way, the global shape of the interpolated low resolution face can be successfully enhanced. Second, we improve the example-based super-resolution method for local high frequency information enhancement. Our proposed algorithm uses the Approximate Nearest Neighbors (ANN) search method to find a number of nearest neighbors for a stack of queries, instead of finding the exact match for each low frequency patch. Median filtering is used to remove the noise from the nearest neighbors in order to enhance the signal. Our proposed algorithm uses a sparse representation and the ANN method to enhance both global face shape and local high frequency information while greatly improving the processing speed, as confirmed empirically.

Face image super-resolution using 2D CCA

In this paper a face super-resolution method using two-dimensional canonical correlation analysis (2D CCA) is presented. A detail compensation step is followed to add high-frequency components to the reconstructed high-resolution face. Unlike most of the previous researches on face super-resolution algorithms that first transform the images into vectors, in our approach the relationship between the high-resolution and the low-resolution face image are maintained in their original 2D representation. In addition, rather than approximating the entire face, different parts of a face image are super-resolved separately to better preserve the local structure. The proposed method is compared with various state-of-the-art super-resolution algorithms using multiple evaluation criteria including face recognition performance. Results on publicly available datasets show that the proposed method super-resolves high quality face images which are very close to the ground-truth and performance gain is not dataset dependent. The method is very efficient in both the training and testing phases compared to the other approaches.

Two-step superresolution approach for surveillance face image through radial basis function-partial least squares regression and locality-induced sparse representation

Face superresolution (SR), or face hallucination, refers to the technique of generating a high-resolution (HR) face image from a low-resolution (LR) one with the help of a set of training examples. It aims at transcending the limitations of electronic imaging systems. Applications of face SR include video surveillance, in which the individual of interest is often far from cameras. A two-step method is proposed to infer a high-quality and HR face image from a low-quality and LR observation. First, we establish the nonlinear relationship between LR face images and HR ones, according to radial basis function and partial least squares (RBF-PLS) regression, to transform the LR face into the global face space. Then, a locality-induced sparse representation (LiSR) approach is presented to enhance the local facial details once all the global faces for each LR training face are constructed. A comparison of some state-of-the-art SR methods shows the superiority of the proposed two-step approach, RBF-PLS global face regression followed by LiSR-based local patch reconstruction. Experiments also demonstrate the effectiveness under both simulation conditions and some real conditions.

Low-Resolution and Low-Quality Face Super-Resolution in Monitoring Scene via Support-Driven Sparse Coding

Monitoring cameras are now widely used to monitor everything from a room in a house to an entire warehouse. However, in real monitoring scenarios, a variety of factors, such as underexposure, optical blurring, defocusing, have an impact on the quality of images, which leads to low-quality and low-resolution (LR) of the individual of interest. Reconstruction of a high-resolution (HR) face image with detailed facial features, from a LR observation based on a set of HR and LR training image pairs, plays an important role in computer vision and face image analysis applications. To super-resolve an HR face given a LR face image, the key issue is how to effectively encode the LR image patch. However, due to stability and accuracy issues, the coding approaches proposed so far are far from satisfactory. In this paper, we present a novel sparse coding method via exploiting the support information on the coding coefficients. According to the distances between the input patch and bases in the dictionary, we first assign different weights to the coding coefficients and then obtain the coding coefficients by solving a weighted sparse problem. Experiments on commonly used databases and some face images on the real monitoring conditions demonstrate that our method outperforms state-of-the-art.

AN EFFECTIVE COLOR FACE RECOGNITION BASED ON BEST COLOR FEATURE SELECTION ALGORITHM USING WEIGHTED FEATURES FUSION SYSTEM

This paper aims to achieve the best color face recognition performance. The newly introduced feature selection method takes advantage of novel learning which is used to find the optimal set of color-component features for the purpose of achieving the best face recognition result. The proposed color face recognition method consists of two parts namely color-component feature selection with boosting and color face recognition solution using selected color component features. This method is better than existing color face recognition methods with illumination, pose variation and low resolution face images. This system is based on the selection of the best color component features from various color models using the novel boosting learning framework. These selected color component features are then combined into a single concatenated color feature using weighted feature fusion. The effectiveness of color face recognition method has been successfully evaluated by the public face databases.

Coupled cross-regression for low-resolution face recognition

Dimensional mismatch between a low-resolution (LR) surveillance face image and its high-resolution (HR) template makes recognition very difficult. A novel method called coupled cross-regression (CCR) is proposed to deal with this problem. Instead of processing in the original observing space directly, CCR projects LR and HR face images into a unified low-embedding feature space. Spectral regression is employed to improve generalization performance and reduce computational complexity. Meanwhile, cross-regression is developed to utilize HR embedding to increase the information of the LR space, thus improving the recognition performance. Experiments on the FERET and CMU PIE face database show that CCR outperforms existing structure-based methods in terms of recognition rate as well as computational complexity.

Face hallucination based on position prior and sparse representation

A face hallucination method based on sparse representation and position prior was proposed,which can obtain the enlargement of a single low-resolution input.Some perspectives of compressed sensing were applied to the method.The high-and low-resolution over-complete atoms were classified according to different positions of face.The low-resolution face image inputs were approximated by the sparse linear combination of the over-complete atoms which were classified.The sparse coefficients were obtained to reconstruct the high-resolution data of certain position.According to their original positions,the generated patches were integrated into a global face.The experimental results illustrate that the proposed method can generate satisfying high-resolution face image using fewer atoms compared to other methods.

A novel face-hallucination scheme based on singular value decomposition

In this paper, an efficient mapping model based on singular value decomposition (SVD) is proposed for face hallucination. We can observe and prove that the main singular values of an image at one resolution have approximately linear relationships with their counterparts at other resolutions. This makes the estimation of the singular values of the corresponding high-resolution (HR) face images from a low-resolution (LR) face image more reliable. From the signal-processing point of view, this can effectively preserve and reconstruct the dominant information in the HR face images. Interpolating the other two matrices obtained from the SVD of the LR image does not change either the primary facial structure or the pattern of the face image. The corresponding two matrices for the HR face images can be constructed in a “coarse-to-fine” manner using global reconstruction. Our proposed method retains the holistic structure of face images, while the learned mapping matrices, which are represented as embedding coefficients of the individual mapping matrices learned from LR-HR training pairs, can be seen as holistic constraints in the reconstruction of HR images. Compared to state-of-the-art algorithms, experiments show that our proposed face-hallucination scheme is effective in terms of producing plausible HR images with both a holistic structure and high-frequency details.

Framework for reliable, real-time facial expression recognition for low resolution images

Automatic recognition of facial expressions is a challenging problem specially for low spatial resolution facial images. It has many potential applications in human–computer interactions, social robots, deceit detection, interactive video and behavior monitoring. In this study we present a novel framework that can recognize facial expressions very efficiently and with high accuracy even for very low resolution facial images. The proposed framework is memory and time efficient as it extracts texture features in a pyramidal fashion only from the perceptual salient regions of the face. We tested the framework on different databases, which includes Cohn–Kanade (CK+) posed facial expression database, spontaneous expressions of MMI facial expression database and FG-NET facial expressions and emotions database (FEED) and obtained very good results. Moreover, our proposed framework exceeds state-of-the-art methods for expression recognition on low resolution images.

Kernel Coupled Cross-Regression for Low-Resolution Face Recognition

Low resolution (LR) in face recognition (FR) surveillance applications will cause the problem of dimensional mismatch between LR image and its high-resolution (HR) template. In this paper, a novel method called kernel coupled cross-regression (KCCR) is proposed to deal with this problem. Instead of processing in the original observing space directly, KCCR projects LR and HR face images into a unified nonlinear embedding feature space using kernel coupled mappings and graph embedding. Spectral regression is further employed to improve the generalization performance and reduce the time complexity. Meanwhile, cross-regression is developed to fully utilize the HR embedding to increase the information of the LR space, thus to improve the recognition performance. Experiments on the FERET and CMU PIE face database show that KCCR outperforms the existing structure-based methods in terms of recognition rate as well as time complexity.

Face Hallucination via Similarity Constraints

In this letter, we present a new face hallucination method based on similarity constraints to produce a high-resolution (HR) face image from an input low-resolution (LR) face image. This method is modeled as a local linear filtering process by incorporating four constraint functions at patch level. The first two constraints focus on checking if the training images are similar to the input face image. The third is defined in the HR face image, which is to impose the smoothness constraint between neighboring hallucinated patches. The final constraint computes the spatial distance to reduce the effect of patches that are far from the hallucinating patch. Experimental evaluation on a number of face images demonstrates the good performance of the proposed method on the face hallucination task.