Budget Constraint Bag-of-Task Based Workflow Scheduling in Public Clouds

Abstract

Bag-of-Tasks (BoT) workflows have appeared in distributed computing platforms such as Spark, MapReduce, and Pegasus. Budget constraints usually exist for these applications. It is crucial to design scheduling algorithms to minimize makespans under budget constraints for BoT workflows. However, most existing workflow algorithms are tailored for general workflows without considering batch structures of Bot-workflows. The main challenge for scheduling BoT workflows is to distribute the budget to different BoTs appropriately considering BoT structures. In this paper, a configuration-and-serialization iterative adjusting based heuristic algorithm (CSIA) is proposed to minimize the makespans under budget constraints. CSIA allocates VM configurations and serial degrees to different BoTs appropriately to decrease the makespan. Experimental results illustrate that the proposal gets shorter makespans on several types of workflow instances than existing algorithms under budget constraints.