3D-KCPNet: Efficient 3DCNNs based on tensor mapping theory

Abstract

As the deep neural networks (DNNs) with satisfied expression ability usually require large scale to gain adequate performance, and deploying large-scale DNNs on resource-limited environments is still a challenge, neural network compression becomes a hot topic nowadays. Among the multiple compression methods, tensor decomposition reveals many specific advantages, such as regular data structure comes from linear algebra, convenient approach of training from scratch, ideal compression ratio, etc. Nevertheless, for some compact neural modules such as two-dimensional/three-dimensional (2D/3D) convolutional kernels, traditional tensor decomposition in the way of approximation has encountered intractable obstacles. Fortunately, some works utilize the tensor mapping approach, which just regards the data structure of tensor decomposition as neural layers, to reconstruct the convolutional kernel into a new lightweight module with several thinner convolutional kernels. The only two flies in the ointment are there lack necessary theories of tensor mapping, and there is still no tensor mapping way to compress high-order three-dimensional convolutional neural networks (3DCNNs). In this paper, we first deeply analyze the tensor mapping theory including convergence and precision, which separately establishes the rationality of tensor mapping and its superiority over the traditional tensor approximation, according to the Lottery Ticket Hypothesis. Then we propose an efficient method termed as 3D-KCPNet, to compress 3DCNNs based on the novel Kronecker canonical polyadic (KCP) tensor decomposition. The proposed method can not only compress the 3D convolutional kernels, but also convert a 3D convolution to efficient 1 × 1 × 1 and 2D depthwise convolutions. The experiments on the video recognition datasets VIVA Challenge, UCF11, UCF50, and UCF101 show that the accuracy of 3D-KCPNet can surpass its original baseline model and the corresponding tensor approximation model.