Face Super-resolution Algorithms Research Articles

A robust face super-resolution algorithm and its application in low-resolution face recognition system

In real-world surveillance scenario, the face recognition (FR) systems pose a lot of challenges due to the captured low-resolution (LR) and noisy probe images. A new face super-resolution (SR) algorithm is proposed to design a recognition model overcoming the challenges of existing FR systems. The proposed SR algorithm inherits the merits of functional-interpolation and dictionary-based SR techniques. The functional interpolation assists in generating more discriminable output, whereas the dictionary-based approach assists in eliminating noise effects from the reconstruction process. Consequently, it produces more discriminable and noise-free high-resolution (HR) images from captured noisy LR probe images, suitable for real-world problems like low-resolution face recognition. The results obtained from the experiments performed on several popular face image datasets including FEI, FERET, and CAS-PEAL-R1 show that the proposed algorithm performs better than all the comparative SR methods.

Robust Face Super-Resolution via Locality-Constrained Low-Rank Representation

Learning-based face super-resolution relies on obtaining accurate a priori knowledge from the training data. Representation-based approaches (e.g., sparse representation-based and neighbor embedding-based schemes) decompose the input images using sophisticated regularization techniques. They give reasonably good reconstruction performance. However, in real application scenarios, the input images are often noisy, blurry, or suffer from other unknown degradations. Traditional face super-resolution techniques treat image noise at the pixel level without considering the underlying image structures. In order to rectify this shortcoming, we propose in this paper a unified framework for representation-based face super-resolution by introducing a locality-constrained low-rank representation (LLR) scheme to reveal the intrinsic structures of input images. The low-rank representation part of LLR clusters an input image into the most accurate subspace from a global dictionary of atoms, while the locality constraint enables recovery of local manifold structures from local patches. In addition, low-rank, sparsity, locality, accuracy, and robustness of the representation coefficients are exploited in LLR via regularization. Experiments on the FEI, CMU face database, and real surveillance scenario show that LLR outperforms the state-of-the-art face super-resolution algorithms (e.g., convolutional neural network-based deep learning) both objectively and subjectively.

Face hallucination based on two-dimensional joint learning

In this paper, a face hallucination method based on two-dimensional joint learning is presented. Unlike the existing works on face super-resolution algorithms that first reshape the image or image patch into 1D vector, in our study the spatial construction of the high resolution (HR) and the low resolution (LR) face image are efficiently maintained in the reconstruction procedure. Enlightened by the 1D joint learning approach for image super-resolution, we propose a 2D joint learning algorithm to map the original 2D LR and HR image patch spaces onto a unified feature subspace. Subsequently, the neighbor-embedding (NE) based super-resolution algorithm can be conducted on the unified feature subspace to estimate the reconstruction weights. With these weights, the initial HR facial image can be generated. To refine further the initial HR estimate, the global reconstruction constraint is exploited to improve the quality of reconstruction result. Experiments on the face databases and real-world face images demonstrate the effectiveness of the proposed algorithm.

Face image super-resolution using 2D CCA

In this paper a face super-resolution method using two-dimensional canonical correlation analysis (2D CCA) is presented. A detail compensation step is followed to add high-frequency components to the reconstructed high-resolution face. Unlike most of the previous researches on face super-resolution algorithms that first transform the images into vectors, in our approach the relationship between the high-resolution and the low-resolution face image are maintained in their original 2D representation. In addition, rather than approximating the entire face, different parts of a face image are super-resolved separately to better preserve the local structure. The proposed method is compared with various state-of-the-art super-resolution algorithms using multiple evaluation criteria including face recognition performance. Results on publicly available datasets show that the proposed method super-resolves high quality face images which are very close to the ground-truth and performance gain is not dataset dependent. The method is very efficient in both the training and testing phases compared to the other approaches.

Face Hallucination Based On Sample Selection Bias Correction

In some learning approach problems, the learning algorithm receives training and test samples drawn according to the same distribution. However, this assumption is not available in practice. In face super-resolution, when the training sample available is biased, it may affect quality of the construction face. In this paper, we proposed a novel method to correct the bias of sample selection in training dataset. First, Active Shape Model is used to get the face shape vectors which contain some information about face contour. Then all faces from Chinese face dataset are classified into certain categories based on Hausdorff Distance by k-means clustering. We correct the sample bias by selecting the most similar faces due to the face shape similarity between samples and test images. Last, the global face reconstruction method based on eigenface is used to achieve satisfied image quality with selected train dataset. Experiments show that the face super-resolution algorithm based on sample selection bias correction can improve the subjective and objective quality of the input low resolution face images compared to traditional eigenface algorithm.

Face Super-resolution based-on Non-negative Matrix Factorization

Principal Component Analysis (PCA) is a classical method which is commonly used for human face images representation in face super-resolution. But the features extracted by PCA are holistic and difficult to have semantic interpretation. In order to synthesize a high-resolution face image with structural details, we propose a face super-resolution algorithm based on non-negative matrix factorization (NMF). This algorithm uses the NMF to obtain structural information representation of face images, and then the target image is regularized by Markov random fields, with maximum a posteriori probability approach. Finally, the steepest descent method is used to optimize NMF coefficient of high-resolution image. Experiment results demonstrate that the NMF-based face super-resolution algorithm performs better than PCA-based algorithms, in the subjective and objective quality.