Channel Attention Module for Efficient Action Recognition

Abstract

For action recognition, 3D CNNs can achieve satisfactory performance, but they are parameter-rich and computationally intensive. Although conventional 2D CNNs have a small computational burden, they do not capture the channel relationships well. To mitigate this issue, we propose a plug-and-play channel attention module that is simple yet effective. Our CA module can perform adaptive refinement of the feature for the given input feature map by capturing channel-wise dependencies. Since the CA module is generic and lightweight, it can be easily integrated into regular convolutional neural networks. We embedded the CA module into TSM and evaluated it by conducting extensive experiments on the commonly used action recognition dataset: Kinetics-400 and gesture recognition dataset: Jester. For Kinetics-400, our CA module was able to improve the top-1 accuracy by 1.24%, 1.46%, 1.43% and 1.53% when using one crop, three crops, five crops and ten crops, respectively. Our experimental results showed that the CA module brings consistent performance improvements, proving the effectiveness of the CA module.