REX-SC: Range-Extended Stochastic Computing Accumulation for Neural Network Acceleration

Abstract

Deep learning has grown in capability and size in recent years, prompting research on alternative computing methods to cope with the increased compute cost. Stochastic computing (SC) promises higher compute efficiency with its compact compute units, but accuracy issues have prevented wide adoption, and accuracy-improving techniques have sacrificed runtime or training performance. In this work, we propose REX-SC -Range-Extended Stochastic Computing Accumulation to deal with the accuracy issues of stochastic computing. By modifying the functionality of OR-based SC accumulation, we increase SC computation accuracy without sacrificing the performance benefits. Our approach achieves a 2X reduction in stream length for the same accuracy compared to SC with OR-based accumulation and a up to 3.6X improvement in energy compared to SC with binary addition. With proper modeling, our approach improves training performance for SC-based neural networks and makes training SC models practical for large datasets like ImageNet.