A two-stream attention mechanisms network

Abstract

In the field of neurology, the concept of attention mechanism is frequently applied to neural networks. Focusing on the application of attention mechanisms in feature map channels and spatial, we propose a two-stream attention networks (called TSAN) based on channel and spatial that can be applied to neural networks. The TSAN network consists of two consecutive parts. The input feature map is first passed through the channel submodule and then through the space sub-module to obtain a feature layer with attention weights. The channel sub-module compresses channel information by using global maximum pooling. The channel sub-module compresses each input feature map into a learnable weight to represent the features of the feature map. The spatial sub-module also uses the global maximum pooling layer to aggregate the location information of the feature maps along the channel axes. By performing spatial compression along the channel axis, the spatial sub-module can generate a corresponding learnable weight for each spatial location. Meanwhile, to further capture the spatial location dependence over long distances, the spatial sub-module will receive inputs from two consecutive feature maps. Since TSAN is a tiny universal block, it can be seamlessly inserted into any CNN architecture with negligible overhead. Also, TSAN can be trained together with the underlying CNN. We validate the neural network with TSAN by conducting extensive experiments using ImageNet, CIFAR-100 datasets. Our experiments show that TSAN can improve the robustness and accuracy of various models for image classification and object detection, which demonstrates the wide applicability of TSAN.