Progressive reconstruction-decoupled face super-resolution framework with controllable knowledge guidance

Abstract

Large-factor face super-resolution is a severely ill-posed problem that requires additional information to assist in image reconstruction. Existing methods improve the super-resolution performance by introducing estimated face or pre-trained generative adversarial network (GAN) priors. However, they ignore the interference between contour and detail reconstruction as well as the errors in pretrained GAN and estimated face priors. To address these problems, we propose a progressive reconstruction-decoupled face super-resolution (PDFSR) framework with controllable knowledge guidance. First, a three-level index (pose, semantics, geometry) is proposed to utilize the high-resolution face texture to construct a lightweight knowledge base robust to bias. Then, the face contours are reconstructed by a basic super-resolution network. Finally, a constraint module is proposed to constrain the output of the knowledge base to provide accurate and controllable explicit knowledge for the reconstruction of facial details. The progressive framework decouples contour and detail reconstruction during training for the reconstructed images to consider both fidelity and perceptual quality. The knowledge base provides accurate and controllable low-dimensional explicit knowledge without estimation or pre-training, which reduces prior errors. Experiments demonstrate that our PDFSR outperforms state-of-the-art face super-resolution methods. Moreover, during the testing phase, faces can be edited by adjusting the explicit knowledge output by the knowledge base to obtain diverse reconstruction results.