Stick Buffer Cache v2: Improved Input Feature Map Cache for Reducing off-chip Memory Traffic in CNN Accelerators

Abstract

Data movement between the Convolutional Neural Network (CNN) accelerators and off-chip memory is critical with respect to the overall power consumption. Minimizing power consumption is particularly important for low power embedded applications. Specific CNN compute patterns offer a possibility of significant data reuse, leading to idea of using specialized on-chip cache memories which enable significant improvement in power consumption. However, due to unique caching pattern present within CNNs, standard cache memories would not be efficient. In this paper novel on-chip cache memory architecture, based on idea of input feature map striping, is proposed, which requires significantly less on-chip memory resources compared to previously proposed solutions. Experiment results show that the proposed cache architecture can reduce on-chip memory size by a factor of 16 or more, while increasing power consumption no more than 15%, compared to some of previously proposed solutions.