A QoS-QoR Aware CNN Accelerator Design Approach

Abstract

Recently powerful convolutional neural network (CNN) accelerators are emerging as energy-efficient solutions for real-time vision/speech processing, recognition and a wide spectrum of approximate computing applications. In addition to the broad applicability scope of such deep learning (DL) accelerators, we found that the fascinating feature of deterministic performance makes them ideal candidates as application-processors in embedded SoCs concerned with real-time processing. However, unlike traditional accelerator designs, DL accelerators introduce the new aspect of design tradeoff between real-time processing [quality of service (QoS)] and computation approximation [quality of result (QoR)] into embedded systems. This paper proposes an elastic CNN acceleration architecture that automatically adapts to the user-specified QoS constraint by exploiting the error-resilience in typical approximate computing workloads. For the first time, the proposed design, including the network tuning-and-mapping software and reconfigurable accelerator hardware, aims to reconcile the design constraint of QoS and QoR, which are respectively, the critical concerns in real-time and approximate computing. It is shown in experiments the proposed architecture enables the embedded system to work flexibly in an expanded operating space, significantly enhances its real-time ability, and maximizes the system energy-efficiency within the user-specified QoS-QoR constraint through self-reconfiguration. Also, we showcase the application of the proposed design approach to lower power image recognition challenge (LPIRC) and how it is employed to forge an energy-efficient solution to the LPIRC contest.