A Fourier domain acceleration framework for convolutional neural networks

Abstract

Acceleration of training and inference of convolutional neural networks (CNNs) plays a significant role in deep learning efforts for large-scale datasets. However, it is difficult to accelerate the training and inference of CNNs based on traditional Fourier domain acceleration frameworks because Fourier domain training and inference are related to many complicated factors, such as the architecture of Fourier domain propagation passes, the representation of the activation function and the design of downsampling operations. A conceptually intuitive, useful and general Fourier domain acceleration framework for CNNs is proposed in this paper. Taking the proposed Fourier domain rectified linear unit (FReLU) as an activation function and the proposed Fourier domain pooling function (FPool) as a downsampling function, a Fourier domain acceleration framework is established for CNNs, and the inverse activation function (FReLU−1) and inverse downsampling function (FPool−1) are further obtained for the backward propagation pass. Furthermore, a block decomposition pipeline is integrated into the Fourier domain forward/backward propagation passes of CNNs to accelerate the training and inference of CNNs. The results show that the proposed acceleration framework can accelerate the training and inference of CNNs by a significant factor without reducing the recognition precision.