Super-resolve Face Images Research Articles

Rethinking Prior-Guided Face Super-Resolution: A New Paradigm With Facial Component Prior.

Recently, facial priors (e.g., facial parsing maps and facial landmarks) have been widely employed in prior-guided face super-resolution (FSR) because it provides the location of facial components and facial structure information, and helps predict the missing high-frequency (HF) information. However, most existing approaches suffer from two shortcomings: 1) the extracted facial priors are inaccurate since they are extracted from low-resolution (LR) or low-quality super-resolved (SR) face images and 2) they only consider embedding facial priors into the reconstruction process from LR to SR face images, thus failing to explore facial priors to generate LR face image. In this article, we propose a novel pre-prior guided approach that extracts facial prior information from original high-resolution (HR) face images and embeds them into LR ones to obtain HF information-rich LR face images, thereby improving the performance of face reconstruction. Specifically, a novel component hybrid method is proposed, which fuses HR facial components and LR facial background to generate new LR face images (namely, LRmix) via facial parsing maps extracted from HR face images. Furthermore, we design a component hybrid network (CHNet) that learns the LR to LRmix mapping function to ensure that the LRmix can be obtained from LR face images in testing and real-world datasets. Experimental results show that our proposed scheme significantly improves the reconstruction performance for FSR.

Face hallucination based on cluster consistent dictionary learning

Face hallucination is a super-resolution technique specially designed to reconstruct high-resolution faces from low-resolution faces. Most state-of-the-art algorithms leverage position-patch prior knowledge of human faces to better super-resolve face images. However, most of them assume the training face dataset is sufficiently large, well cropped or aligned. This paper, proposes a novel example-based face hallucination method, based on cluster consistent dictionary learning with the assumption that human faces have similar facial structures. In this method, the paired face image patches are firstly labelled as face areas including eyes, nose, mouth and other parts, as well as non-face areas without requiring the training face images cropped and aligned. Then, the training patches are clustered according their labels and textures. The cluster consistent dictionary is learned to represent the low-resolution patches and the high-resolution patches. Finally, the high-resolution patches of the input low-resolution face image can be efficiently generated by using the adjusted anchored neighbourhood regression. As utilizing the labelled facial parts prior knowledge, the proposed method represents more details in the reconstruction. Experimental results demonstrate that the authors' algorithm outperforms many state-of-the-art techniques for face hallucination under different datasets.

Dual-Path Deep Fusion Network for Face Image Hallucination.

Along with the performance improvement of deep-learning-based face hallucination methods, various face priors (facial shape, facial landmark heatmaps, or parsing maps) have been used to describe holistic and partial facial features, making the cost of generating super-resolved face images expensive and laborious. To deal with this problem, we present a simple yet effective dual-path deep fusion network (DPDFN) for face image super-resolution (SR) without requiring additional face prior, which learns the global facial shape and local facial components through two individual branches. The proposed DPDFN is composed of three components: a global memory subnetwork (GMN), a local reinforcement subnetwork (LRN), and a fusion and reconstruction module (FRM). In particular, GMN characterize the holistic facial shape by employing recurrent dense residual learning to excavate wide-range context across spatial series. Meanwhile, LRN is committed to learning local facial components, which focuses on the patch-wise mapping relations between low-resolution (LR) and high-resolution (HR) space on local regions rather than the entire image. Furthermore, by aggregating the global and local facial information from the preceding dual-path subnetworks, FRM can generate the corresponding high-quality face image. Experimental results of face hallucination on public face data sets and face recognition on real-world data sets (VGGface and SCFace) show the superiority both on visual effect and objective indicators over the previous state-of-the-art methods.

Joint Super-Resolution and Alignment of Tiny Faces

Super-resolution (SR) and landmark localization of tiny faces are highly correlated tasks. On the one hand, landmark localization could obtain higher accuracy with faces of high-resolution (HR). On the other hand, face SR would benefit from prior knowledge of facial attributes such as landmarks. Thus, we propose a joint alignment and SR network to simultaneously detect facial landmarks and super-resolve tiny faces. More specifically, a shared deep encoder is applied to extract features for both tasks by leveraging complementary information. To exploit representative power of the hierarchical encoder, intermediate layers of a shared feature extraction module are fused to form efficient feature representations. The fused features are then fed to task-specific modules to detect landmarks and super-resolve face images in parallel. Extensive experiments demonstrate that the proposed model significantly outperforms the state-of-the-art in both landmark localization and SR of faces. We show a large improvement for landmark localization of tiny faces (i.e., 16 × 16). Furthermore, the proposed framework yields comparable results for landmark localization on low-resolution (LR) faces (i.e., 64 × 64) to existing methods on HR (i.e., 256 × 256). As for SR, the proposed method recovers sharper edges and more details from LR face images than other state-of-the-art methods, which we demonstrate qualitatively and quantitatively.

Contourlet Transform based Multi-scale Fusion Network for Face Hallucination

Recently, Convolutional neural network (CNN) has achieved great success in the field of face hallucination. However, such approaches usually generate blurry and over-smoothed Super Resolution (SR) results, and the performance suffers from degradation when super-resolve a very Low Resolution (LR) face image. To solve these problems, this paper proposes an contourlet transform based accurate CNN architecture, namely Multi-scale Fusion CNN (MSFC), which are able to reconstruct a High Resolution (HR) face image from a very low resolution input. First of all, we present multi-scale fusion CNN (MSFC) to fully detect and exploit features from LR inputs. And then, we formulate the SR problem as the prediction of contourlet transform coefficients, which is able to make MSFC further capture the texture details for super-resolve face images. Extensive qualitative and quantitative experiments show that the proposed method is capable of preserving details and achieves superior SR performance compared to the state-of-the-art methods.

Residual Attribute Attention Network for Face Image Super-Resolution

Facial prior knowledge based methods recently achieved great success on the task of face image super-resolution (SR). The combination of different type of facial knowledge could be leveraged for better super-resolving face images, e.g., facial attribute information with texture and shape information. In this paper, we present a novel deep end-to-end network for face super resolution, named Residual Attribute Attention Network (RAAN), which realizes the efficient feature fusion of various types of facial information. Specifically, we construct a multi-block cascaded structure network with dense connection. Each block has three branches: Texture Prediction Network (TPN), Shape Generation Network (SGN) and Attribute Analysis Network (AAN). We divide the task of face image reconstruction into three steps: extracting the pixel level representation information from the input very low resolution (LR) image via TPN and SGN, extracting the semantic level representation information by AAN from the input, and finally combining the pixel level and semantic level information to recover the high resolution (HR) image. Experiments on benchmark database illustrate that RAAN significantly outperforms state-of-the-arts for very low-resolution face SR problem, both quantitatively and qualitatively.

Imagining the Unimaginable Faces by Deconvolutional Networks.

We tackle the challenge of constructing 64 pixels for each individual pixel of a thumbnail face image. We show that such an aggressive super-resolution objective can be attained by taking advantage of the global context and making the best use of the prior information portrayed by the image class. Our input image is so small (e.g., pixels) that it can be considered as a patch of itself. Thus, conventional patch-matching-based super-resolution solutions are unsuitable. In order to enhance the resolution while enforcing the global context, we incorporate a pixel-wise appearance similarity objective into a deconvolutional neural network, which allows efficient learning of mappings between low-resolution input images and their high-resolution counterparts in the training data set. Furthermore, the deconvolutional network blends the learned high-resolution constituent parts in an authentic manner, where the face structure is naturally imposed and the global context is preserved. To account for the possible artifacts in upsampled feature maps, we employ a sub-network composed of additional convolutional layers. During training, we use roughly aligned images (only eye locations), yet demonstrate that our network has the capacity to super-resolve face images regardless of pose and facial expression variations. This significantly reduces the requirement of precisely face alignments in the data set. Owing to the network topology we apply, our method is robust to translational misalignments. In addition, our method is able to upsample rotational unaligned faces with data augmentation. Our extensive experimental analysis manifests that our method achieves more appealing and superior results than the state of the art.

Face Recognition in Uncontrolled Condition: Can Compressive Sen sing and Super-Resolution Meet the Challenge?

This paper is concerned with face recognition under uncontrolled conditions, e.g. at a distance in surveillance scenarios, and post-rioting forensic, whereby captured face images are severely degraded/blurred and of low resolution. This is a tough challenge due to many factors including difficulties in determining a model for image degradation that encompasses a range of realistic capturing conditions. We present the results of our investigations into recently developed Compressive Sensing (CS) theory to develop scalable face recognition schemes using a variety of overcomplete dictionaries that construct super-resolved face images from any input low-resolution degraded face image. We shall demonstrate that deterministic as well as non-deterministic dictionaries that do not involve the use of face image information but satisfy some form of the Restricted Isometry Property (RIP) used for CS can achieve face recognition accuracy levels as good a, if not better than, those achieved by dictionaries, proposed in the literature, that are learnt from face image databases using elaborate procedures. We shall elaborate on how this approach helps in crime fighting and terrorism.

Computationally efficient algorithm for face super-resolution using (2D) 2 -PCA based prior

Super-resolution algorithms typically transform images into 1D vectors and operate on these vectors to obtain a high-resolution image. In this study, the authors first propose a 2D method for super-resolution using a 2D model that treats images as matrices. We then apply this 2D model to the super-resolution of face images. Two-directional two-dimensional principal component analysis (PCA) [(2D) 2 -PCA] is an efficient face representation technique where the images are treated as matrices instead of vectors. We use (2D) 2 -PCA to learn the face subspace and use it as a prior to super-resolve face images. Experimental results show that our approach can reconstruct high quality face images with low computational cost.

Super-Resolution of Face Images Using Kernel PCA-Based Prior

We present a learning-based method to super-resolve face images using a kernel principal component analysis-based prior model. A prior probability is formulated based on the energy lying outside the span of principal components identified in a higher-dimensional feature space. This is used to regularize the reconstruction of the high-resolution image. We demonstrate with experiments that including higher-order correlations results in significant improvements

Hallucinating multiple occluded face images of different resolutions

Learning-based super-resolution has recently been proposed for enhancing human face images, known as “face hallucination”. In this paper, we propose a novel algorithm to super-resolve face images given multiple partially occluded inputs at different lower resolutions. By integrating hierarchical patch-wise alignment and inter-frame constraints into a Bayesian framework, we can probabilistically align multiple input images at different resolutions and recursively infer the high-resolution face image. We address the problem of fusing partial imagery information through multiple frames and discuss the new algorithm’s effectiveness when encountering occluded low-resolution face images. We show promising results compared to those of existing face hallucination methods from both simulated facial database and live video sequences.