Efficient Flow-Based Scheduling for Geo-Distributed Simulation Tasks in Collaborative Edge and Cloud Environments

Abstract

Edge computing is a good complement to cloud computing for deploying large-scale geo-distributed simulation applications, which are very sensitive to the communication delay among different simulation components (also called tasks in this paper) and users. We mainly focus on the efficient scheduling of simulation components in collaborative edge and cloud environments. As components should be deployed jointly with the consideration of capacity constraints of hosts, it is actually an NP-complete multi-dimensional bin packing problem. Meanwhile, dynamic changes of component and host states require the low deployment latency of scheduling algorithms. Unfortunately, most of the existing schedulers for modern clusters are queue-based, in which tasks are scheduled sequentially, thus lacking the ability to process tightly coupled tasks jointly. Other batching-based placement algorithms are usually time-consuming. This paper describes Pond, a novel flow-based scheduler with the awareness of interactions among tasks and users as well as heterogeneous multi-dimensional resources. First, characteristics of distributed simulation tasks are analysed and the scheduling problem is formulated as a min-cost max-flow (MCMF) problem over the flow network by mapping the communication overhead among tasks and users to the costs of arcs in the network. Considering the inherent defects of existing flow-based schedulers in dealing with multi-dimensional resources, a new method based on dominant resource is proposed and some problem specific heuristics are also designed. Extensive simulation experiments based on Alibaba production trace and some random synthetic parameters are conducted. Results show that Pond can reduce the average communication cost for each task significantly in a quite low deployment latency compared with some baselines.