Dimension-aware attention for efficient mobile networks

Abstract

Recently, attention mechanisms have shown great potential in improving the performance of mobile networks. Typically, they involve 2D symmetric convolution operations or generate 2D attention maps. However, such manners usually introduce high computational cost and large memory consumption, increasing the computational burden of mobile networks. To address this problem, we propose a novel lightweight attention mechanism, called Dimension-Aware Attention (DAA) block, by modeling the intra-dependencies of each dimension of the input feature map. Specifically, we factorize the channel and spatial attention by three parallel feature vector encoding branches, where stacked 1D asymmetric convolution operations can be naturally leveraged to capture large receptive fields. In this way, channel-aware, horizontal-aware, and vertical-aware attention vectors are extracted to effectively encode multi-dimensional information and greatly reduce the computational complexity of mobile networks. Experiments on multiple vision tasks demonstrate that our DAA block achieves better accuracy against state-of-the-art attention mechanisms with much lower computational operations. Our code is available at https://github.com/rymo96/DAANet.