Splatter: An Efficient Sparse Image Convolution for Deep Neural Networks

Abstract

Deep neural network (DNN) based approaches, such as deep convolutional neural network (CNN), have achieved highly accurate results in many fields (e.g., computer vision, etc.) at the cost of a huge number of parameters and high computational workloads. The parameters require large memory capacity and memory access time which cause a migration problem to embedded devices. Pruning techniques can reduce the DNN complexity, but it brings sparsity in the matrix which causes computational inefficiency and performance loss. The reasons for the inefficiency include the reduced data reuse opportunities, waste of memory bandwidth, and computational irregularity. Applying sparse matrix formats can help to reduce inefficiency with a regular computational pattern of the sparse matrix (e.g., Compressed Sparse Row), but it has a limitation to improve the efficiency in data reuse and memory bandwidth. In this paper, we propose the Splatter which is an efficient sparse image convolution technique for DNN. In the convolution sweep, non-zero input data is multiplied by each kernel element and the outcomes will be accumulated into the output. We focus on reducing memory access and increasing data reuse. The performance of our proposed technique is compared to naïve convolution method with a dense matrix format and CSR format as well. Our experimental results with sparse images, which are 50%~90% sparsity, show that the Spatter can improve the execution time of image convolution by 25%~81% with dense matrix, and 49%~90% with a CSR matrix format. Additionally, an average reduction in input data accesses of 97% is observed using the proposed convolution method.