Kernel Sharing in the Channel Dimension to Improve Parameters Efficiency

Abstract

In many real-time applications, the deployment of deep neural networks is constrained by its high computational cost. Therefore, reducing redundancy and designing efficient neural networks are widely concerned. In this paper, we propose that the convolution kernel can be shared in the channel dimension, which can be seen as the same convolution kernel sliding on the channel. A novel efficient convolution is proposed by using group convolution and kernel sharing, which is named as kernel sharing group convolution (KSGC). KSGC is more parameter efficient than standard convolution and general group convolution (GGC). Subsequently, the KSGCNet is constructed by utilizing KSGC. We compare KSGCNet with ResNet, MobileNet and ResNeXt on CIFAR dataset. The results show that KSGCNet has higher accuracy than ResNet and MobileNet. KSGCNet is more flexible and parameter efficient than ResNeXt because of two flexible hyper-parameters, the number of input feature maps in each group and kernel sliding stride. Moreover, the two hyperparameters can make a trade-off between accuracy and model size.