AutoMap: Automatic Mapping of Neural Networks to Deep Learning Accelerators for Edge Devices

Abstract

Emerging deep neural networks (DNNs) have been emerging in applications (object detection, automatic speech recognition, etc.) deployed on edge devices. To improve the energy efficiency of edge devices, domain specific deep learning accelerators (DLAs) are designed with limited on-chip resources. The manifold DLA designs and evolving DNN topologies bring challenges for applications mapping and scheduling on hardware resources. In this paper, we propose an automatic DNN mapping framework named AutoMap given the hardware backend information. Firstly, a computational graph representation called Extended Directed Weighted Graph (EDWG) is proposed, which realizes unified expression for both spatial and temporal network inter-layer connections. Secondly, an associated partitioner is implemented for splitting an EDWG into subEDWGs, which incorporates the on-chip memory constraint and facilitates weight data reuse on chip. Lastly, a dynamic memory allocation strategy is utilized to alleviate the feature storing burden introduced by the multivarious network sizes and connections. Compared to the baseline mapping methods, experimental results show that our proposed automatic mapping framework can help to speedup the execution of several DNNs on state-of-the-art DLAs, ranging from 1.27x to 3.45x. The utilization of the PE array can increase from 20% to 64%.