Experimental Analysis of Algorithms for Coflow Scheduling

Abstract

Modern data centers face new scheduling challenges in optimizing job-level performance objectives, where a significant challenge is the scheduling of highly parallel data flows with a common performance goal e.g., the shuffle operations in MapReduce applications. Chowdhury and Stoica [6] introduced the coflow abstraction to capture these parallel communication patterns, and Chowdhury et al. [8] proposed effective heuristics to schedule coflows efficiently. In our previous paper [18], we considered the strongly NP-hard problem of minimizing the total weighted completion time of coflows with release dates, and developed the first polynomial-time scheduling algorithms with O1-approximation ratios. In this paper, we carry out a comprehensive experimental analysis on a Facebook trace and extensive simulated instances to evaluate the practical performance of several algorithms for coflow scheduling, including our approximation algorithms developed in [18]. Our experiments suggest that simple algorithms provide effective approximations of the optimal, and that the performance of the approximation algorithm of [18] is relatively robust, near optimal, and always among the best compared with the other algorithms, in both the offline and online settings.