Performance Evaluation of State-of-the-Art Edge Computing Devices for DNN Inference

Abstract

Fast Deep Neural Network (DNN) inference is nowadays possible thanks to the recent significant advancements in computing platforms and DNN frameworks. The advent of the Internet of Things, among other factors, leads to the massive deployment of edge computing devices, whose features and performance are very diverse. In order to help designers identify the hardware platforms and DNNs that best suit their target embedded applications, this paper presents a DNN inference performance analysis for state-of-the-art edge devices. These include Graphical Processing Units, Tensor Processing Units and Field-Programmable Systems-on-Chip. Different versions of the MobileNet and Inception DNNs and different frameworks are considered in the analysis.