A Novel Adaptive Computation Offloading Strategy for Collaborative DNN Inference over Edge Devices

Abstract

With the breakthrough of deep learning technology and the rapid growth of Internet of Things (IoT'), a fast-increasing number of artificial intelligence (AI) applications are being widely used in many smart IoT systems. Taking the UAV last mile delivery system as an example, UAVs require the assistance of various AI applications such as object detection and route planning during the delivery process. However, provisioning AI services over the cloud computing environment are facing many challenges such as network congestion and high latency. To tackle these challenges, edge computing has emerged as a new computing paradigm where computing resources are provisioned closer to the end devices from the network edge. In this paper, combined with the edge computing-based UAV last mile delivery scenario, we focus on the computation offloading problem for deep learning-based applications at the inference stage. A novel computation offloading strategy for multi-device collaborative pipelining processing of deep neural networks (DNNs) tasks called CoDNN is proposed. First, when the DNN inference task is generated, CoDNN will predict the execution time for each layer of the DNN and model the process of DNN inference. Second, CoDNN will use the offloading partitioning algorithm based on Particle Swarm Optimization (PSO) to determine the target computing resources and assign them to appropriate workloads. Specifically, we explore the impact of different computing environments on the performance of CoDNN and show its advantages over other representative computation offloading strategies.