Abstract
The rapid development of machine learning is enabling a plenty of novel applications, such as image and speech recognition for embedded and mobile devices. However, state-of-the-art deep learning models like convolutional neural networks (CNNs) are demanding so much on-chip storage and compute resources that they cannot be smoothly handled by low-power mobile or embedded systems. In order to fit large CNN models into mobile or more cutting-edge devices for IoT or cyberphysics applications, we proposed an efficient on-chip memory architecture for CNN inference acceleration, and showed its application to in-house single-instruction multiple-data structure machine learning processor. The redesigned on-chip memory subsystem, Memsqueezer, includes an active weight buffer and data buffer set that embraces specialized compression methods to reduce the footprint of CNN parameters (weights) and activation data, respectively. Memsqueezer buffer can compress the data and weight set according to the dataflow in computation, and it also includes a built-in redundancy detection mechanism that actively scans through the working-set of CNNs to boost their inference performance by eliminating the computation redundancy in CNN models. In our experiments, it is shown that the CNN processors with Memsqueezer buffers achieve more than $2{\times }$ performance improvement and reduces 85% energy consumption on average over the conventional buffer design with the same area budget.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.