A Non-Local Enhanced Network for Image Restoration

Yuan Huang,Xueming Qian,Yujie Dun,Zan Chen,Xingsong Hou

doi:10.1109/access.2022.3148201

Yuan Huang, Xueming Qian + Show 3 more

Open Access

https://doi.org/10.1109/access.2022.3148201

Copy DOI

Abstract

Non-local modules have been widely studied in image restoration (IR) tasks since they can learn long-range dependencies to enhance local features. However, most existing non-local modules still focus on extracting long-range dependencies within a single image or feature map. On the other hand, most IR methods simply employ a single type of non-local module in the network. A combination of various types of non-local modules to enhance local features can be more effective. In this paper, we propose a batch-wise non-local module to explore richer non-local dependencies within images. Furthermore, we combine various non-local extractors (different attention modules) with the proposed batch-wise non-local module as the Enhanced Batch-wise Non-local Attentive module (EBNA). Besides exploring richer non-local information, we build the Non-local and Local Information extracting Block (NLIB), in which we combine the EBNA with DEformable-Convolution Block (DECB) to utilize richer non-local and adaptive local information. Finally, We embed the NLIB within a U-net-like structure and build the Non-local Enhanced Network (NLENet). Extensive experiments on synthetic image denoising, real image denoising, JPEG artifacts removal, and real image super resolution tasks demonstrate that our proposed network achieves state-of-the-art performance on several IR benchmark datasets.

Highlights

Image restoration is a classic computer vision task that aims to restore high-quality image from its various degradation
Extensive experiments on synthetic image denoising, real image denoising, JPEG artifacts removal and real image super resolution tasks show that the proposed model achieves state-of-the-art performance
(1) we propose a batch-wise non-local module to explore the relevance among images; (2) based on the proposed batch-wise non-local module, we propose Enhanced Batch-wise Non-local Attentive module (EBNA), which provides enriched non-local features from various types of non-local modules; (3) to fully utilize the non-local and local information, we propose Non-local and Local Information extracting Block (NLIB) built upon EBNA and DEformable-Convolution Block (DECB)

Summary

Introduction

Image restoration is a classic computer vision task that aims to restore high-quality image from its various degradation. It has been widely applied in many practical applications, such as medical image processing [1], [2], surveillance [3]– [5], synthetic aperture radar (SAR) image processing [6]–[8], image compression [9], and so on

Methods

Results

Conclusion