A Novel Pooling Method for Regularization of Deep Neural networks

Abstract

Dropout has been introduced as a powerful regularization approach to preventing overfitting problem in deep neural networks, particularly in deep convolutional neural networks(DCNNs). A number of methods have been designed recently to improve and generalize the dropout technique. These methods include spectral dropout which achieves improved generalization and avoids overfitting by eliminating noisy and weak Fourier domain coefficients of the neural network activations. On the other hand, a pooling process plays a crucial role in deep convolutional neural networks, which serves to reduce the dimensionality of processed data for decreasing computational cost as well as for avoiding overfitting and enhancing the generalization capability of the network. For this reason, we focus on the pooling layer, and we propose a new pooling method called Spectral Dropout Pooling, by applying the Spectral dropout technique in the pooling region, in order to avoid overfitting problem, as well as to enhance the generalization ability of DCNNs. Experimental results on several image benchmarks show that Spectral Dropout Pooling outperforms the existing pooling methods in classification performance as well as is effective for improving the generalization ability of DCNNs. Moreover, we show that Spectral Dropout Pooling combined with other regularization methods, such as batch normalization, is competitive with other existing methods in classification performance.