Neighbour feature attention-based pooling

Abstract

In modern convolutional neural networks (CNNs), the pooling layer is seen as one of the primary layers for building the CNN model, which effectively downscales the spatial size of feature maps to reduce memory consumption. Several types of pooling operations, such as average pooling, max pooling, and strided convolution, fail to capture the spatial dependence between the pooling region feature and its neighbour features. In this paper, we propose a simple but effective attention-based pooling method called Neighbour Feature Attention-Based Pooling (NFP), which integrates neighbour features of the pooling region to keep semantic continuity across multiple layers. NFP adopts attention weights encoding with neighbour features by depthwise convolution, which effectively directs local spatial pooling for learning discriminative features. Compared to other pooling methods, the proposed method generates more discriminative features directed by neighbour information of the pooling region. The experiments results show that it consistently improves the performance across various backbone architectures on image classification tasks.