Slardar: Scheduling information incomplete inter-datacenter deadline-aware coflows with a decentralized framework

Abstract

Nowadays, geographically distributed edge clusters and datacenters generate large volumes of wide-area network (WAN) traffic. Therefore, it is critical to schedule inter-datacenter flows, especially coflows that imply application-level semantics, to improve the performance of geo-distributed applications. However, coflows often have a mixture of soft and hard deadline requirements, which cannot be efficiently guaranteed by existing methods. In this paper, a decentralized deadline-ware coflow scheduling framework, called Slardar, is proposed. It schedules inter-datacenter coflows while considering both hard and soft deadline requirements. However, it is challenging to design such a framework without complete coflow information. Hence, we first model the intra-coflow bandwidth allocation problem as a mean-field game where each datacenter aims to minimize its own cost with only local coflow information. We further prove its mean-field equilibrium is also the optimal bandwidth allocation strategy and propose a deep reinforcement learning based algorithm to find an approximate optimal numerical solution. Then, we investigate the offline inter-coflow bandwidth allocation problem by modeling it as a linear programming problem, and an online greedy heuristic algorithm is proposed to find an approximate optimal solution. Finally, we compare Slardar with state-of-art methods by large-scale simulation experiments. Experimental results show that our proposed method can greatly improve the transmission revenue and increase the soft and hard deadline guarantee ratios by 7.5% and 10% at least.