Workflow Makespan Minimization for Partially Connected Edge Network: A Deep Reinforcement Learning-Based Approach

Abstract

The ever advances in wireless communication and mobile networks have brought novel workflow-formed applications, such as virtual reality and live-streaming, to our daily life. Arousing a growing need for workflow execution efficiency. An edge network is widely considered a promising way of bridging the gap between intensive resource demand and the limited computation capabilities of mobile terminals. However, when an edge network is partially connected, ordinary workflow scheduling algorithms suffer degradations as the data transmission time is prolonged. In this paper, we address the challenge of workflow makespan minimization in a partially connected edge network. Contrary to the general assumptions of a fully connected edge network, the edge servers under discussion are partly interconnected but can be reached within limited hops by using different paths. Here, the placement of interdependent tasks and selection of routing paths are two major factors that influence the makespan. We first propose a critical path analysis based dynamic task sorting algorithm to determine the scheduling order of tasks. Then the path quality is introduced as a reflection of path availability and is employed as the major indicator in selecting disjoint subpaths. We further model the workflow scheduling process into a Markov decision process and propose a reinforcement learning–based workflow embedding (RLWE) scheme to minimize the makespan of the workflow. With the fine-trained agent, the proposed scheme can coordinate the demand of computing resources and routing paths of interdependent tasks and provide a near-optimal makespan of the workflow. Numerical results validate the feasibility of our proposed scheme as its performance exceeds existing baselines with an improved quality of service in terms of makespan.