EdgeWise: Energy-efficient CNN Computation on Edge Devices under Stochastic Communication Delays

Abstract

This article presents a framework to enable the energy-efficient execution of convolutional neural networks (CNNs) on edge devices. The framework consists of a pair of edge devices connected via a wireless network: a performance and energy-constrained deviceDas the first recipient of data and an energy-unconstrained deviceNas an accelerator forD. DeviceDdecides on-the-fly how to distribute the workload with the objective of minimizing its energy consumption while accounting for the inherent uncertainty in network delay and the overheads involved in data transfer. These challenges are tackled by adopting the data-driven modeling framework of Markov Decision Processes, whereby an optimal policy is consulted byDinO(1) time to make layer-by-layer assignment decisions. As a special case, a linear-time dynamic programming algorithm is also presented for finding optimal layer assignment at once, under the assumption that the network delay is constant throughout the execution of the application. The proposed framework is demonstrated on a platform comprised of a Raspberry PI 3 asDand an NVIDIA Jetson TX2 asN. An average improvement of 31% and 23% in energy consumption is achieved compared to the alternatives of executing the CNNs entirely onDandN. Two state-of-the-art methods were also implemented and compared with the proposed methods.