Boomerang: On-Demand Cooperative Deep Neural Network Inference for Edge Intelligence on the Industrial Internet of Things

Abstract

With the revolution of smart industry, more and more Industrial Internet of Things (IIoT) devices as well as AI algorithms are deployed to achieve industrial intelligence. While applying computation-intensive deep learning on IIoT devices, however, it is challenging to meet the critical latency requirement for industrial manufacturing. Traditional wisdom resorts to the cloud-centric paradigm but still works either inefficiently or ineffectively due to the heavy transmission latency overhead. To address this challenge, we propose Boomerang, an on-demand cooperative DNN inference framework for edge intelligence under the IIoT environment. Boomerang exploits DNN right-sizing and DNN partition to execute DNN inference tasks with low latency as well as high accuracy. DNN right-sizing reshapes the amount of DNN computation via the early-exit mechanism so as to reduce the total runtime of DNN inference. DNN partition adaptively segments DNN computation between the IoT devices and the edge server in order to leverage hybrid computation resources to achieve DNN inference immediacy. Combining these two keys, Boomerang carefully selects the partition point and the exit point to maximize the performance while promising the efficiency requirement. To further reduce the manual overhead of model profiling at the install phase, we develop an advanced version of Boomerang with the DRL model, achieving end-to-end automatic DNN inference plan generation. The prototype implementation and evaluations demonstrate the effectiveness of Boomerang on both versions in achieving efficient edge intelligence for IIoT.