Avg-topk: A new pooling method for convolutional neural networks

Abstract

The pooling layer is a layer used in Convolutional Neural Networks (CNN) that takes the output feature map of the previous convolutional layer and reduces the feature maps to smaller sizes. Furthermore, in CNN the pooling layer is one of the layers that determines the success of the model. The pooling layer, reduces the spatial dimension of a CNN, greatly reducing the learning time and computational cost of the model. The most common pooling methods are maximum and average pooling. Due to the fact that the pooling strategy reduces the amount of feature maps and model parameters, it is crucial to preserve the dominant information. In this study, a cost-effective new pooling method approach is proposed. The proposed pooling method is used by calculating the weighted average of the dominant features. The proposed pooling model has been developed to address the shortcomings of maximum pooling and average pooling. The proposed new Avg-TopK pooling model takes the pixels with the highest interaction as much as the specified K number and averages them. In this study, the performances of several pooling strategies for gray and RGB picture classification in 3 different datasets were compared and analyzed in depth. Extensive experiments have demonstrated that the Avg-TopK pooling method achieves significantly higher image classification accuracy than conventional pooling methods. It has been observed that using the AVG-TopK method in transfer learning models leads to much more successful results. Furthermore, studies in the literature have compared based on the performance metrics and it has been seen that the proposed method produces more successful outcomes. In research conducted on datasets using this method, the accuracy achieved for the CIFAR-10 dataset was 6.28% and 16.62% according to the maximum pooling and the average pooling, respectively. For the CIFAR-100 dataset, the accuracy rate increased by 7.76% compared to the maximum pooling and by 25% compared to the average pooling.