Resource-Efficient Distributed Deep Neural Networks Empowered by Intelligent Software-Defined Networking

Abstract

Contemporary machine learning methods have evolved from conventional algorithms to deep neural networks (DNNs) that are computation- and data- intensive. Thus, they are suitable to be deployed in the cloud that can offer high computational capacity and scalable resources. However, the cloud computing paradigm is not optimal for delay- and energy-sensitive applications. To mitigate these problems, a battery of distributed DNNs have been proposed to allow a fast inference with device-edge-cloud synergy. Furthermore, although distributed deployment of DNNs on real communication networks is an important research topic, the legacy network architecture cannot meet the requirements of these distributed deep neural networks due to the complicated management and manual configuration, etc. To cope with these requirements, we develop a novel and explicit Intelligent Software Defined Networking (ISDN) that aims to manage the bandwidth and computing resources across the network via the SDN paradigm. We first identify the difficulties of deploying distributed intelligent computing in the current network architecture. Then, we explain how to address these problems by introducing the ISDN architecture. Specifically, we develop a dynamic routing method to enable Quality-of-Service (QoS) communication based on the SDN paradigm and propose a Markov Decision Process (MDP) based dynamic task offloading model to achieve the optimal offloading policy of DNN tasks. We develop a simulation platform based on Mininet to measure its performance advantages over traditional architectures. Extensive experimental results show that compared with the traditional network architecture, our architecture based on the SDN paradigm can perform better in terms of both network throughput and resource utilization.