Joint optimization of tasks placement and routing to minimize Coflow Completion Time

Abstract

Coflow is defined as the parrellel flows between two successive computation stages of data-parallel jobs. Reducing Coflow Completion Time (CCT) is important to improve the performance of data-parallel applications in geo-distributed datacenter networks. CCT is influenced by two factors: tasks placement and coflow routing. Previous works that optimize only one factor are insufficient in reducing CCT.In this paper, we explore the joint optimization of tasks placement and coflow routing to reduce CCT. We formulate the joint optimization of a single coflow as a Mixed Integer Non-Linear Programming problem, and propose an approximate algorithm PRO with an approximation ratio of (1 + ε). Moreover, we propose algorithms to optimize multiple coflows scheduling offline and online. Through extensive experiments, we demonstrate that our proposed algorithms have superior performance in reducing CCT or average CCT compared with other algorithms.