Information-Agnostic Traffic Scheduling in Data Center Networks with Asymmetric Topologies

Abstract

As more and more applications are deployed in data centers, they rely on high performance data center networks (DCNs) to meet users’ increasing quality of the experience (QoE) requirements. Hence, minimizing the average flow completion time (FCT) has been one of the most important goals for DCNs. However, existing traffic scheduling methods assume either prior knowledge of flows (i.e., sizes and deadlines) or symmetric topologies (i.e., Fat-Tree or Bcube). In practice, it is difficult to obtain the information of flows. Moreover, even with symmetric topology design, the DCNs will become asymmetric due to the inevitable link failure and congestion. In this case, it is a great challenge to minimize the average FCT in DCNs. In this paper, we propose a flowlet based information-agnostic traffic scheduling mechanism. The key idea of our method is leveraging multiple priority queues in switches to demote the priority of flows dynamically on the flowlet level. More specifically, the priority of a flow will be demoted according to the number of flowlets it has sent, which follows the shortest job first discipline. We formulate the average FCT minimization problem as a nonlinear Sum-of-Ratios problem and design two heuristic methods to derive the sub-optimal demotion thresholds. Experiment results show that our method can reduce the average FCT by up to 15.35% with a realistic workload, as compared to the state-of-the-art traffic scheduling methods.