Guided Cascaded Super-Resolution Network for Face Image

Lin Cao,Yanan Guo,Kangning Du,Jiape Liu,Tao Wang

doi:10.1109/access.2020.3025972

Lin Cao, Yanan Guo + Show 3 more

Open Access

https://doi.org/10.1109/access.2020.3025972

Copy DOI

Abstract

The image super-resolution algorithm can overcome the imaging system’s hardware limitation and obtain higher resolution and clearer images. Existing super-resolution methods based on convolutional neural networks(CNN) can learn the mapping relationship between high-resolution(HR) and low-resolution(LR) images. However, when the reconstruction target is a face image, the reconstruction results often have problems that the face area is too smooth and lacks details. We propose a guided cascaded face super-resolution network, called guided cascaded super-resolution network (GCFSRnet). GCFSRnet takes the LR image and a high-quality guided image as inputs, and it consists of a pose deformation module and a super-resolution network. Firstly, the pose deformation module converts the guide image’s posture into the same as the low-resolution face image based on 3D fitting and 3D morphable model (3DMM). Then, the LR image and the deformed guide image are used as input of the super-resolution network. The super-resolution networks are formed by a cascade of two layers of networks, which extract different features. During the reconstruction process, the guide image can provide real facial details and help generate subtle facial textures. The cascade structure of a super-resolution network can gradually extract features and restore different levels of image details. The experimental results on the CASIA Web Face and CelebA datasets show that the proposed method can generate facial images with clear outlines and rich details, which are superior to other state-of-the-art methods such as SRResNet, SRGAN, VDSR, DBPN, etc.

Highlights

Super-resolution reconstruction is one of the classic computer vision problems, which aims to recover high-frequency details from low-resolution videos or images
The GCFSRnet consists of two parts: a pose deformation module noted as WarpNet (Mwarp) and a super-resolution network noted as RecNet (G1, G2)
In order to ensure that the SR network to obtain enough information from the input, we select another image of the same identity as the LR image to be the guide image, which can provide real high-frequency information for the reconstruction

Summary

INTRODUCTION

Super-resolution reconstruction is one of the classic computer vision problems, which aims to recover high-frequency details from low-resolution videos or images. UR-DGN uses the approximately aligned frontal face image to train the model and resolve the artifacts in the reconstruction results caused by the input image’s very low resolution. Each layer of the SR network takes the distorted guide image together with the low-resolution image as a common input to generate subtle facial textures. In the face correction algorithms proposed in recent years, Hassner et al [20] uses a unique and fixed 3D face model to approximate the fitted shape of input faces This method is very effective for the human face’s frontal area but will cause severe texture loss and artifacts on the contour and nearcontour surfaces. OVERVIEW OF GCFSRnet In this work, we propose a novel guided cascaded face super-resolution network called GCFSRnet. The discriminator randomly selects an image to determine whether it is a real high-quality image

THE POSE DEFORMATION MODULE

THE SUPER-RESOLUTION NETWORK

LOSS FUNCTIONS

CONCLUSION