An Adaptive Scheduling Mechanism for Analytical Workflow Model

Abstract

With the recent development of big data and cloud computing, more and more applications of data analytics emerged. Cloud workflow is a good tool to orchestrate analytical tasks, which is called analytical workflow. In this paper, we focus on the resource scheduling for analytical workflow. As there exist multiple instance for each task when executing, the execution time of workflow is dynamical change with the resource. First of all, we model the performance of analytical workflows executed in cloud and formulate the scheduling problem that minimizing the execution time with budget constraint. Then, we propose an adaptive scheduling algorithm and take machine learning algorithm as case study to illustrate the performance of our algorithm.