Scheduling Bulk File Transfers with Start and End Times

Abstract

The advancement of optical networking technologies has enabled e-science applications that often require transport of large volumes of scientific data. In support of such data-intensive applications, we develop and evaluate control plane algorithms for scheduling bulk file transfers, where each transfer has a start time and an end time. We formulate the scheduling problem as a special type of the multicommodity flow problem in which the time line is divided into uniform slices. Bandwidth is allocated to each job on every slice with the objective of improving throughput of the concurrent transfers. We study the effectiveness of using multiple time slices and multiple paths for each file transfer. We show that using a small number of paths per job is generally sufficient to achieve near optimal throughput with a practical execution time.