Universal super-resolution for face and non-face regions via a facial feature network

Abstract

This study proposes a fusion method for universal super-resolution (SR) that produces good results for both face and non-face regions. We observed that most general-purpose SR networks fail to sufficiently reconstruct facial features. In addition, face-specific networks degrade the performance on non-face regions. To reconstruct face regions well, face-specific SR networks are trained by employing a facial feature network. Then, to preserve the performance on the non-face regions, a region-adaptive fusion that uses both face-specific and general-purpose networks is proposed. In the fusion stage, a face detection algorithm is included, and detected masks are smoothed to avoid boundary artefacts. The results indicate that the proposed method significantly improves the performance on the face region and delivers a similar performance on the non-face region. In addition, the experimental results indicate that additional computations can be considerably reduced by sharing the front layers of the networks.