GFNet: A Gradient Information Compensation-Based Face Super-Resolution Network

Shengxiang Luo,Jinbo Lu

doi:10.1109/access.2022.3143499

Abstract

Face super-resolution (FSR) is defined as the generation of high-resolution face images from low-resolution face images. Existing FSR approaches usually improve the performance by combining deep learning with additional tasks such as face parsing and landmark prediction. However, the additional data requires manual labeling, and facial landmark heatmaps and parsing maps cannot represent the intrinsic geometric structure of facial components. In this paper, we introduce a FSR network based on gradient information compensation named GFNet, which consists of feature residual blocks (FRBs) and gradient extraction blocks (GEBs). Specifically, the GEB constructs pixel-level gradient maps directly from the feature maps without requiring data labels and extracts gradient features to compensate for the missing high-frequency components in the face features; the FRB extracts the face features in the network. Furthermore, we introduced a feature fusion mechanism between the GEB and the FRB, which fuses the face features with the gradient features. We evaluate the performance of proposed network on the two public datasets: CelebA-HQ dataset and Helen dataset. Experimental results show that the proposed method is able to reconstruct fine face images, which outperforms the other state-of-the-art methods such as SRResnet, FSRNet, and MSFSR.

Highlights

Face super-resolution (FSR), known as face hallucination, generates high-resolution (HR) face images from lowresolution (LR) face images
Unlike the single image super-resolution (SISR) tasks [4]–[8], the difficulty of FSR tasks lies in the recovery of facial components, i.e., eye and nose
Lu: GFNet: Gradient Information Compensation-Based Face Super-Resolution Network fine facial details of faces, we propose a gradient extraction block (GEB), which constructs pixel-level gradient maps directly from face features; second, we employ a feature residual block (FRB) to extract face features in the network

Summary

INTRODUCTION

Face super-resolution (FSR), known as face hallucination, generates high-resolution (HR) face images from lowresolution (LR) face images. The network extracts more facial structure information by introducing these additional tasks, there exist the following drawbacks: 1) it requires manual labeling of data for the additional tasks; 2) estimating face priors from low-resolution inputs is itself a difficult task; and 3) facial landmark heatmaps and parsing maps cannot represent the intrinsic geometric structure of facial components, such as the nose bridge. Part-based methods extract facial components, which requires the detection of facial landmarks in the LR image The performance of both methods degrades significantly in FSR tasks with a high upscale factor. The proposed GFNet uses gradient maps with pixel-level accuracy to compensate for the missing high-frequency components in face features, which means that we can obtain more semantic information even in very low-resolution images.

GRADIENT EXTRACTION BLOCK

FEATURE FUSION MECHANISM

CONCLUSION