Multi-Level Difference Repair Architecture for Face Hallucination

Abstract

Facial hallucination (FH) aims to construct super-resolution (SR) face images based on the corresponding low-resolution (LR) thumbnails. The desired results of FH require correct identity and photorealistic image quality, however, developing a method that satisfies both requirements remains challenging. Although the lifelike faces predicted by some existing methods, such as PULSE, are impressive, the loss of original identity is unacceptable. In view of the deficiency of PULSE, in this letter we incorporate a conditional constraint to its unsupervised pixel recovery process. Specifically, we embed a PULSE generator within the Laplacian pyramid framework and install a corresponding pre-trained conditional adversarial net (CGAN) at each pyramid level to predict the difference between the reconstructed SR image obtained by PULSE and the ground truth. The difference is added to the reconstructed SR image at each level to obtain a new SR image that is closer to the ground truth, such a process is repeated across the subsequent levels until a full-resolution version is yielded. Extensive experiments have demonstrated that our architecture can obtain a realistic high-resolution (HR) face with a correct identity, which is not possible obtained by other FH methods before.