CoEdge: Exploiting the Edge-Cloud Collaboration for Faster Deep Learning

Abstract

Recently a great number of ubiquitous Internet-of-Things (IoT) devices have been connecting to the Internet. With the massive amount of IoT data, the cloud-based intelligent applications have sprang up to support accurate monitoring and decision-making. In practice, however, the intrinsic transport bottleneck of the Internet severely handicaps the real-time performance of the cloud-based intelligence depending on IoT data. In the past few years, researchers have paid attention to the computing paradigm of edge-cloud collaboration; they offload the computing tasks from the cloud to the edge environment, in order to avoid transmitting much data through the Internet to the cloud. To present, it is still an open issue to effectively allocate the deep learning task (i.e., deep neural network computation) over the edge-cloud system to shorten the response time of application. In this paper, we propose the latency-minimum allocation (LMA) problem, aimed at allocating the deep neural network (DNN) layers over the edge-cloud environment while the total latency of processing this DNN can be minimized. First, we formalize the LMA problem in general form, prove its NP-hardness, and present an insightful characteristic of feasible DNN layer allocations. Second, we design an approximate algorithm, called CoEdge, which can handle the LMA problem in polynomial time. By exploiting the communication and computation resources of the edge, CoEdge greedily selects the beneficial edge nodes and allocates the DNN layers to the selected nodes by a recursion-based policy. Finally, we conduct extensive simulation experiments with realistic setups, and the experimental results show the efficacy of CoEdge in reducing the deep learning latency compared to two state-of-the-art schemes.