S-DenseNet: A DenseNet Compression Model Based on Convolution Grouping Strategy Using Skyline Method

Abstract

DenseNet has strong expressiveness in many computer vision tasks, but the complexity of its model makes it difficult to deploy to devices with limited computing resources. In this paper, we propose S-DenseNet, a compact model of DenseNet which makes the extracting features of DenseNet more comprehensively and reduces the parameter redundancy at the same time. Firstly, we investigate the parameter redundancy of DenseNet and show the possibility of compressing DenseNet. For example, we reduced its parameters by 10% as keeping in the same accuracy. Secondly, we propose an algorithm based on the Skyline method for converting standard convolution into group convolution automatically. It can choose the filter weights automatically and restore the standard convolution filter with a high sparsity one called S-Conv. Finally, we combine S-Conv with dense connections and formed a more effective network architecture, that is S-DenseNet. Experiments show that in Cifar10, Cifar100 and SVHN datasets, we use only 40% parameters and 20% FLOPs of DenseNet, and the accuracy is improved by 1%. Compared with the compression models of DenseNet and other CNNs, we use less complexity in the model and achieve higher or similar accuracy. On ImageNet dataset, compared with lightweight CNN models, traditional CNNs and their compression models, we achieve the higher or similar Top-1 accuracy with less complexity.