Bottleneck-Stationary Compact Model Accelerator With Reduced Requirement on Memory Bandwidth for Edge Applications

Abstract

State-of-the-art compact models such as MobileNets and EfficientNets are structured using a linear bottleneck and inverted residuals. Hardware architecture using a single dataflow strategy fails to balance the required memory bandwidth with the given computational resources. This work presents a heterogeneous dual-core accelerator that performs a block-wise pipelined process as a unit using a bottleneck-stationary (BS) dataflow. The BS greatly relieves the requirement on DRAM bandwidth and on-chip SRAM capacity. A look-behind-only attention is also proposed as a co-optimized algorithm. Compared to the state-of-the-art hardware scheme, the proposed accelerator demonstrates a reduction of 1.8- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.9\times $ </tex-math></inline-formula> in latency and 2.2- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3\times $ </tex-math></inline-formula> in energy consumption, respectively.For verification, the accelerator with a 16-bit integer precision was implemented using 28nm CMOS process. Measurements show energy efficiencies of 0.5-to-3.75 TOPS/W in a supply voltage range of 0.55-to-1.15V.