Formula omitted]-norm low-rank linear approximation for accelerating deep neural networks

Abstract

In this paper, we develop a L1-norm based low-rank matrix approximation method to decompose large high-complexity convolution layers into a set of low-complexity convolution layers with low-ranks to accelerate deep neural networks. Based on the alternating direction method (ADM), we derive a mathematical solution for this new L1-norm based low-rank decomposition problem. Our experimental results on public datasets, including CIFAR-10 and ImageNet, demonstrate that this new decomposition scheme outperforms the recently developed L2-norm based nonlinear decomposition method, which achieved the state-of-the-art performance.