Dual attention per-pixel filter network for spatially varying image deblurring

Abstract

Spatially varying motion deblurring has recently witnessed substantial progress due to the development of deep neural network. However, most existing CNN-based methods involve two major shortcomings: (1) The CNN weights are space-sharing, and these methods thus ignore the properties of complex spatially variant blurs which vary from pixel to pixel in natural blurry images. (2) Stacked convolution layers with a large kernel or recurrent neural networks (RNNs) cannot capture the global contextual dependence of features, they thus cannot exploit the relationship between different blur pixels at a distance. To solve these problems, we propose a new dual attention per-pixel filter network (DAPFN). First, we develop a multiscale per-pixel filter network (MSPFN) to learn a specific deblurring mapping for different blur pixels, which predicts the per-pixel spatially adaptive convolution kernel for each blur pixel in the input blurry image of different scales and restores the clean pixel by performing channel-wise spatially adaptive convolution with the local neighborhood pixels. Second, we develop a dual attention enhanced residual network (DAERN) to capture the global contextual dependence of the blurry images, which introduces a dual attention (DA) module consisting of the spatial self-attention module (SSA) and channel self-attention module (CSA). The fusion of the two attention modules helps to further improve the deblurring performance. Third, we propose a new receptive field selection (RFS) block to learn the nonlinear characteristics of spatially variant blurs, which enables the adaptive fusing of the features with different receptive fields and effectively enhances the network nonlinear representation ability. The experimental results on GOPRO dataset indicate that the average PSNR and SSIM of the proposed method reached 31.8455 and 0.9231, respectively. The results of extensive experiments pertaining to spatially varying image deblurring demonstrate that the proposed method outperforms the state-of-the-art image deblurring methods.