Stacked deformable convolution network with weighted non-local attention and branch residual connection for image quality assessment

Abstract

Convolutional neural networks are data-driven Image Quality Assessment (IQA) models based on the convolutional operation in a rectangular window. Deformable convolutions with learnable receptive fields can efficiently extract structural features of irregular objects in an image. By superimposing deformable convolutions, a plug-and-play module is designed to obtain information for irregular geometric shapes. To selectively fuse shallow and deep features, we propose a weighted non-local attention (WNLA) module with the input and output of self-attention in a weighted manner. This paper proposes a dual branch residual full-reference IQA network that combines weighted non-local attention and stacked deformable convolution. The proposed network was trained on PIPAL dataset and tested on LIVE and TID2013. The cross-dataset evaluation shows that the network has a competitive generalization ability. Ablation experiments indicate that the proposed modules can effectively improve the performance of the network. Comparative experiments reveal that our network is superior to existing excellent networks. The codes for training, test and visualization are available at: https://github.com/Pengchang-haha/SDCN.git.