SADD: A Novel Systolic Array Accelerator with Dynamic Dataflow for Sparse GEMM in Deep Learning

Abstract

AbstractNowadays, deep learning is prevalent in many fields. The primary workload in deep learning is the General Matrix-matrix Multiplication (GEMM). The TPU is the state-of-the-art GEMM accelerator. However, it does not support sparsity. In this paper, we design and implement the SADD, a systolic array accelerator that supports sparsity and dynamic dataflow. First, we propose the Group-Structure-Maintained Compression (GSMC). Then, based on the GSMC, we propose the Sparsity-supported Weight Stationary Dataflow (SWS) and Sparsity-supported Input Stationary Dataflow (SIS) to exploit the sparsity for systolic arrays. Finally, by combining the SIS and SWS, we propose the Sparsity-supported Dynamic Dataflow (SDD), which can change dataflow according to the computing environment. The experimental results show that the SDD in the SADD perform efficiently in any computing environment. When running the AlexNet, the performance of the SADD is \(2 \times\) better than the TPU. In addition, the SADD brings only a small additional hardware overhead.KeywordsDeep learningSparsityDynamic dataflowSystolic array