An efficient DNN splitting scheme for edge-AI enabled smart manufacturing

Abstract

Deep Neural Network (DNN)-based IoT solutions are enabling automation in smart manufacturing. However, the execution of these compute-intensive solutions in real/near-real time is still a challenging issue. Edge-AI solutions utilize the partial computational offloading-based DNN splitting schemes, which employ collaborative computing to minimize the execution time of compute-intensive DNN task(s). Single-task DNN splitting solutions did not consider multi-task aspects and multi-task-based splitting schemes suffer from additional issues that deteriorate their performance in multi-task smart manufacturing scenarios. This work proposes a Task Aware DNN splitting (TADS) scheme that addresses the above issues. TADS collectively utilizes the number and type of tasks, computing, and communication resources to select the DNN splitting policy from the policy pool to minimize average task execution time in smart manufacturing scenarios. It executes policy pool update, candidate policy selection, and optimal policy selection phases iteratively to determine the final DNN splitting policy. Three DNN models (including a product quality inspection use-case) are evaluated under various scenarios by varying the number of tasks, task inter-arrival time, and bandwidth. The simulation results and comparative analysis with ECN-only, ES-only, DNN-off, and Greedy based DNN splitting approaches under various scenarios in terms of average task execution time indicate the efficacy of the TADS scheme. The scheme is also evaluated on a hardware-based testbed for vision-based quality inspection use-case to indicate the utility and efficiency of proposed work in multi-task smart manufacturing scenarios. A NodeJS based web API is developed for vision-based quality inspection application.