Conflict-Resilient Incremental Offloading of Deep Neural Networks to the Edge of Smart Environment

Zhongmin Chen,Zhiwei Xu,Limin Liu,Jie Tian,Yujun Zhang,Jianxiong Wan,Wenyu Zhang

doi:10.1155/2021/9985006

Abstract

Novel smart environments, such as smart home, smart city, and intelligent transportation, are driving increasing interest in deploying deep neural networks (DNN) in edge devices. Unfortunately, deploying DNN at resource-constrained edge devices poses a huge challenge. These workloads are computationally intensive. Moreover, the edge server-based approach may be affected by incidental factors, such as network jitters and conflicts, when multiple tasks are offloaded to the same device. A rational workload scheduling for smart environments is highly desired. In this work, we propose a Conflict-resilient Incremental Offloading of Deep Neural Networks at Edge (CIODE) for improving the efficiency of DNN inference in the edge smart environment. CIODE divides the DNN model into several partitions by layer and incrementally uploads them to local edge nodes. We design a waiting lock-based scheduling paradigm to choose edge devices for DNN layers to be offloaded. In detail, an advanced lock mechanism is proposed to handle concurrency conflicts. Real-world testbed-based experiments demonstrate that, compared with other state-of-the-art baselines, CIODE outperforms the DNN inference performance of these popular baselines by 20%to 70%and significantly improves the robustness under the insight of neighboring collaboration.

Highlights

In recent years, deep neural networks (DNN) have been widely used in edge environments
From smart home [1] and smart city [2] to automatic driving [3], everyday objects around us are becoming “smarter,” by integrating terminals related to daily life and realizing a smart, safe, convenient, artistic, and energy-saving environment based on DNNs
We propose the Conflict-resilient Incremental Offloading of Deep Neural Networks at Edge (CIODE) for improving the DNN inference efficiency at the edge smart environment

Summary

Introduction

Deep neural networks (DNN) have been widely used in edge environments. Based on the neural network execution graph (in Section 3), we propose the CIODE It can solve the optimization problem of how to determine the DNN layers that need to be uploaded, the upload order, and the collaborative devices during DNN collaborative computation. According to the above information, CIODE builds a multiple edge nodes DNN collaborative execution graph, determines DNN layers that need to be uploaded, and records the corresponding target nodes and sorts based on delay improvement. E purpose of offloading DNN computing tasks is minimizing the execution overhead to improve the quality of service (QoS) of DNN applications To this end, we design the CIODE Planning algorithm (Algorithm 1), which first divides the DNN model into partitions and collects information, including estimated waiting time, network speed, available nodes, and prediction files.

Evaluation

Experimental Setup

Result

Conclusions