Cascaded face super‐resolution with shape and identity priors

Abstract

AbstractDespite impressive progress in face super‐resolution (SR), it is an open challenge to reconstruct a reliable SR face that preserves authentic facial characteristics. Here, the problem of super‐resolving low‐resolution (LR) faces to high‐resolution (HR) ones is addressed. To tackle the ill‐posed nature of face SR, the cascaded super‐resolution network (CSRNet) is proposed to utilize shape and identity priors jointly and progressively, the first to explore multiple priors. Specifically, CSRNet adopts a cascaded structure to transform an LR face to HR face progressively via multiple stages. At each stage, CSRNet forces its output face image to match both the shape priors and identity priors extracted from the ground‐truth HR face. The shape priors estimated in one stage are merged into the inputs of its subsequent stage to provide rich information for the face SR. To generate realistic yet discriminative faces, the cascaded super‐resolution generative adversarial network (CSRGAN) is also proposed to incorporate the adversarial loss and identification loss into CSRNet. Extensive experiments on popular benchmarks show that the CSRNet and CSRGAN outperform existing face SR state‐of‐the‐art methods, both quantitatively and qualitatively, and detailed ablation studies show the advantage of this method.