An Acceleration Framework for Super-Resolution Network via Region Difficulty Self-adaption

Abstract

With the development of deep neural network (DNN), many DNN-based super-resolution (SR) models have achieved state-of-the-art (SOTA) performance. But the applications of these SOTA models are limited by the high computational and memory costs. We observed that different image regions have different difficulties in the process of SR reconstruction. However, current DNN-based SR models process different types of regions equally, and thus involve much redundant computation in the regions with low SR difficulty. To address this limitation, this paper proposes a general acceleration framework for SR networks, which first distinguishes the SR difficulty of image regions, and then applies large model and light model for difficult regions and easy regions respectively. Experimental results demonstrated that the proposed acceleration framework can accelerate SOTA SR networks with 2–4 times without reducing of quality performance.