Channel-Modulated Multibranch Convolutional Neural Network

Abstract

Depth-wise convolution has become very popular recently, owing to research on efficient convolutional networks. However, it performs convolution in each channel separately, therefore, decreases the representational power of features. In this thesis, we take a systematic research on the architecture and shortcomings of depth-wise convolution. We observe that combining the benefits of multibranch depth-wise operation, shared channel modulation, and feature fusion brings us considerable improvements on both computation accuracy and convolution efficiency. This simple yet effective idea enables us to propose a novel channel-modulated multibranch convolution (CMMB-Conv). In our approach, we first use a multibranch depth-wise operation on the input feature maps to increase the channel width. Next, we gather the spatial information of feature maps from multiple branches through Max and Average Pooling layers. Followed by this, a shared hidden neural perceptron is employed to modulate the inter-channel relationship of the feature maps. Finally, we concatenate the multibranch-modulated feature maps and fuse them by using the point-wise convolution. Compared with feature maps extracted by the depth-wise separable convolution, the feature maps resulting from our CMMBConv have strong representation capability, improving the accuracy of existing MobileNets on both ImageNet and CIFAR classification. Extensive experimental results on the ImageNet2012 have shown that our CMMBConv can jointly improve accuracy and efficiency. Specifically, the Top-1 accuracy is increased by 3.3% to 73.91% in contrast with that of the depth-wise convolution. Meanwhile, the number of parameters and floating-point operations are reduced by 29.02% and 30.77%, respectively, compared with the standard convolution.