Image content-dependent steerable kernels

Abstract

Attention mechanism plays an essential role in many tasks such as image classification, object detection, and instance segmentation. However, existing methods typically assigned attention weights to feature maps of the previous layer. The kernels in current layer remained static during the inference stage. To explicitly model the dependency of individual kernel weights on image content at the inference stage, this work proposed attention weight block (AWB) that converts kernels to be steerable to the content in a test image. Specifically, AWB computes a set of on-the-fly coefficients according to the feature maps of the previous layer and applies the coefficients to the kernels in current layers, which makes them steerable. AWB kernels emphasize or suppress the weights of certain kernels depending on the content of input samples and hence significantly improve the feature representation ability of deep neural networks. The proposed AWB is evaluated on various datasets, and experimental results show that steerable kernels in AWB outperformed the state-of-the-art attention approaches when embedded in the architecture for classification, object detection, and semantic segmentation tasks. It outperforms ECA by 1.1% and 1.0% on CIFAR-100 and Tiny ImageNet datasets, respectively, for image classification task; outperforms CornerNet-Lite by 1.5% on COCO2017 dataset for object detection task; and outperforms FCN8s by 1.2% on SBUshadow dataset for semantic segmentation task.