Tensor neural networks via circulant convolution

Abstract

Enough computation and storage resources have made Convolutional Neural Networks (CNNs) prosperous in many fields, but it is still challenging to deploy these models on mobile or embedded devices. Designing effective and efficient convolution modules can vastly reduce the number of parameters and computational complexity of CNNs to favor the deployment. This paper proposes a Circulant Convolution (CC) established on a tensor algebra operator of t-product. Specifically, a feature map can be parameterized by multilinear operation to make the channels positively coupled, and then a more compact parameter space and stronger parameterization capability can be simultaneously obtained. Replacing standard spatial convolution with CC can effectively reduce almost 9 times the number of network parameters and FLOPs, and the impact on accuracy is much smaller than other well-designed convolution modules for lightweight networks. CC can be embedded into existing network architectures as a plug-and-play module, and its topology structure can be easily extended to high-dimensional data. To favor the use of CC in CNNs, a circulant convolution module (CCM), also known as the bottleneck of CC, is also designed by combining CC and pointwise convolution. In further, a lightweight network CCMNet is constructed based on incorporating CC and CCM into an existing lightweight backbone. Extensive experiments on the benchmark dataset CIFAR10/100, ImageNet, Lung-CXR, and MS COCO, demonstrate that our proposed CC could significantly improve the performance of several advanced lightweight networks, and CCMNet is competitive with the start-of-the-art portable neural networks.