Adaptive Salience Preserving Pooling for Deep Convolutional Neural Networks

Abstract

Pooling is an important operation in deep convolutional neural networks (CNNs). In the learning process of CNNs, the pooling layer needs to preserve feature information while significantly reduce computation amount. The conventional pooling methods, such as average pooling and max pooling cannot preserve feature information effectively in pooling layers. Therefore, a more effective and adaptive pooling mechanism is needed for pooling layers. In this paper, we propose a novel pooling mechanism to preserve feature information effectively. The novel pooling method considers the value distribution of the activations within pooling regions and operates on salient activations accordingly. This pooling method can be realized by clustering the activations in pooling regions into salient and trivial activations, and use salient activations only to compute pooling output. Compared with traditional average and max pooling methods, our pooling method is more adaptive and more likely to preserve feature information in pooling layers. Experimental results on several benchmark image datasets show that our adaptive salience preserving (ASP) pooling method outperforms existing pooling methods. Furthermore, our proposed ASP pooling can provide a middle zone between average pooling and max pooling. By using ASP pooling in the right stages of CNNs, we could significantly improve the network performance. For example, the performance of VGG network can be improved by changing the last two pooling layers to be ASP pooling.