Improving pooling method for regularization of convolutional networks based on the failure probability density

Abstract

This research proposes an improved pooling method for regularized convolutional neural network (CNN). This pooling method intends to assign failure probability density (FPD) values to pixel points in image feature domains after projection of image eigenvectors from high to low dimensions to maintain the relationship of highly dimensional image features. As a result, feature mapping of some samples approximated failure probability, and residual samples featured low risk of failing. Optimization was implemented according to this idea and was realized by setting the threshold value of FPD to reserve high-quality features. Different from traditional pooling method based on CNN, the pooling method proposed in this study is based on failure probability theory, and it was used as basis for construction of CNN structure. Image classification tests on three kinds of image datasets (CIFAR-10, CIFAR-100, and SVHN) were respectively conducted. Afterward, comparisons were made on experimental accuracy and speed obtained through three relatively popular pooling methods (i.e., dropout-pooling, maxout-pooling, and stochastic- pooling). Research results indicated that pooling model based on failure probability theory featured scientific derivation without the need for empirical parameters and presented the most accurate results in experiments on three kinds of image in training data and test data. This model also presented high efficiency in speed of model training, proving its robustness.