Efficient Compression Methods for Wire-Spread-Based Stochastic Computing Deep Neural Networks

Abstract

To accelerate the computation of deep neural networks (DNNs), their hardware implementations have been extensively explored. Stochastic computing (SC) is recognized as an efficient computing paradigm that can be adopted to design low-complexity architectures for DNNs. However, most of the reported SC-DNNs suffer from long latency. Recently, a novel SC scheme called wire spreading has been proposed to address the issue of latency, delivering fully parallel SC-DNN architectures. This brief aims to further reduce the hardware complexity of the wire-spread-based SC-DNNs (WSSC-DNNs). By investigating the bit distribution, we first propose a consecutive bit compression (CBC) method without introducing any accuracy loss. Moreover, another method called approximate compression (AC) is presented to reduce the number of wires at the expense of acceptable accuracy loss. These two compression methods are compatible and therefore can be utilized simultaneously. In the experiment for SqueezeNet on CIFAR10, it shows that the proposed methods can save up to 70.6% area cost and 67.0% power consumption.