Towards effective learning for face super-resolution with shape and pose perturbations

Abstract

Recent development of convolutional neural networks (CNNs) has activated a lot of studies and boosted the performance greatly on image super-resolution. This paper addresses the issue of face super-resolution, which attracts a lot of interests in the photographic industry. We propose to make use of the face-specific priors to enhance the performance of face super-resolution with the convolutional neural networks. Classical facial prior models represent the 2D facial shape in a compact low-dimensional space expressed by principal components. Here, we impose perturbations on the low-dimensional space and generate face samples with novel appearance. First, we conduct 2D facial image perturbations through 2D facial landmarks. Then, we carry on the study with perturbations on 3D facial landmarks. Facial pose and shape are perturbated to generate novel appearances of a single 2D facial image. These novel facial samples are then fed into the training process of the convolutional neural networks for face super-resolution. The experimental results demonstrate that the proposed method is adaptable to various networks and achieves superior performance for the face super-resolution task.