Characterizing the Performance of Deep Learning Inference for Edge Video Analytics

Abstract

AbstractReal-time video analytics is a killer application for edge computing, however, it has not been fully understood how much edge resource is required to support compute-intensive video analytics tasks at the edge. In this paper, we conduct an in-depth measurement study to unveil the resource requirement of deep learning (DL) inference for edge video analytics (EVA). We measure the DL inference time under various resource combinations, with different physical resources (e.g., processor types, core numbers, memory), video resolutions, and DL inference models. It is observed that the relationship between DL inference time and resource configuration is complicated, which cannot be captured by a simple model. Considering the coupling effects, we model the resource requirements of the DL inference by utilizing tensor factorization (TF) method. Our TF-based DL inference model can match well with the observations from field measurements. Different from previous models, our model is completely explainable and all the components have their physical meanings. Especially, it is possible to match the top three components extracted from the TF-based model with the practical DL functional layers, namely, convolutional layer, pooling layer, and fully-connected layer. The measurement results show that the top three components contribute approximately 50%, 25%, and 10% time for DL inference execution, which provides clear instruction on fine-grained DL task scheduling for edge video analytics applications.KeywordsDeep learning inferenceModelingTensor factorizationEdge video analytics