Activation pruning of deep convolutional neural networks

Abstract

Deep neural networks (DNNs) have risen to prominence, in the last few years, thanks to their very good performance on different classification and recognition tasks. However, their implementations suffer from long latency caused by the complexity of the network. Recently, many hardware implementations were introduced to accelerate the processing time of DNNs, and in particular of convolutional layers. While they can easily meet the timing constraint of real-time applications for small networks, complex models such as VGG and VGG-like networks are still out of reach. In this paper, we propose an technique to prune the neurons of each convolutional layer, also called activations, which directly contribute to the latency. Comparing networks with the same number of activations, we show that the activation-pruned networks perform better than the unpruned networks in terms of misclassification error.