Image Deblurring Based on Efficient Transformer and Multi-scale CNN

Abstract

The study of image deblurring techniques in dynamic scenes is a high-profile research direction. Recently, given the excellence of Convolutional Neural Networks (CNNs) in extracting feature information, it has become a common and effective practice to utilize them for blurred image restoration work. However, CNN can only model local information and has a limited receptive field, which inhibits the deblurring effect. Transformer can model global information, so it can be combined with CNN to expand the receptive field and enhance the deblurring effect. Unfortunately, as the spatial resolution of the input image increases, the computational complexity of the Transformer increases dramatically, showing a trend of square-level growth, which makes it difficult to cope with the task of processing high-resolution images. To address the above problems, this paper proposes an image deblurring network based on efficient Transformer and multi-scale CNN called ET-MIMO-UNet. The local spatial features are extracted using multi-scale CNN and embedded into the global characteristics of the Transformer, modelling both local and global information. To solve the problem of difficult training due to large image size and to improve the computational efficiency, an efficient Transformer layer (ETL) is designed, which contains a multi-dconv head transposed attention (MHTA) and a gated-dconv feed-forward network(GFFN). In addition, a multi-layer feature fusion block (MFFB) is introduced to fuse full-scale features and reduce feature loss. On the GoPro test dataset, compared with the MIMO-UNet base network, the PSNR of the three models of ET-MIMO-UNet is improved by 0.39dB, 0.54dB, and 0.66dB, respectively; ET-MIMO-UNet reduces the number of parameters by half compared with MPRNet. The experimental data fully proved that the method demonstrated a significant processing effect in coping with the image-blurring problem in dynamic scenes.