Local to non-local: Multi-scale progressive attention network for image restoration

Abstract

Image restoration (IR) tasks aim to form a balance between complex textures and spatial details. To this end, the combination of local and non-local attention mechanisms has been well studied in recent years. However, existing local attention-based modules ignore the interaction between channel and spatial attention, while non-local attention operations solely focus on short-range or long-range dependency. To overcome these problems, a novel multi-scale progressive attention network is proposed in this paper, which is termed as MPANet. The proposed MPANet is composed of two parts, a local multi-scale feature extractor and a window-dilation self-attention module. In the local multi-scale feature extractor, a single-scale feature enhancement strategy is designed to model the correlation between channel and spatial dimensions, and a multi-scale feature fusion strategy is applied to further exchange contextual information across all the scales. Furthermore, the window-dilation self-attention is introduced to establish global representation while preserving local details. Experimental results on four IR tasks demonstrate that the proposed MPANet outperforms the state-of-the-art methods in both quantitative results and visual perception.