Load balance based workflow job scheduling algorithm in distributed cloud

Abstract

As the scale of the geo-distributed cloud increases and the workflow applications become more complex, the system operation is more likely to cause the waste of resources and excessive energy consumption. In this paper, a workflow job scheduling algorithm based on load balancing is proposed to efficiently utilize cloud resources. Firstly, the execution time of the jobs on the cloud is estimated based on the state of the cloud. Then, a queuing model is established for each cloud to minimize the total response time of the system. Finally, the job scheduling problem in a geo-distributed cloud can be transformed into the minimum system response time problem. Moreover, a workflow task scheduling algorithm based on the shortest path algorithm is proposed to minimize all task completion time and energy consumption. Firstly, the directed acyclic graph (DAG) of the tasks can be converted into the hypergraph according to the execution order of the tasks. Then, the k-path hypergraph partition is performed with the balance of the hypergraph. Finally, the Dijkstra shortest path algorithm is used to find the optimal task scheduling strategy which is performed on each hypergraph partition. The experimental results indicate that our proposed workflow scheduling method can effectively utilize cloud resources and reduce system energy consumption. Moreover, the applicability of the proposed effective scheduling strategy is shown in the scenarios of new media live video application.