Classification-Based Diverse Workflows Scheduling in Clouds

Abstract

Cloud workflow scheduling is a typical combinatorial optimization problem and becomes more challenging due to the increasing diversity of workflows. However, current research employs the same scheduling strategy on diverse workflows. In fact, a scheduling strategy may perform well on one workflow but poorly on other workflows owning to the unique characteristics of each workflow. Therefore, in practical applications, selecting suitable scheduling strategies for diverse workflows is a critical issue. To solve it, this paper investigates a diverse workflows scheduling problem and presents a classification-based workflow scheduling framework, which includes workflow parser, workflow classifier, workflow scheduler, resource manager and workflow status tracker, to manage and schedule diverse workflows using suitable strategies. Based on the framework, we propose a classification-based workflow scheduling algorithm (CWSA) to optimize the economic cost of workflow execution under deadline constraints. We conduct the experiments using diverse workflow instances randomly generated from five types of real-world workflows to evaluate the proposed CWSA approach. The results demonstrate the superiority of CWSA compared with the state-of-the-art approaches. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Diverse workflows (i.e., many workflows with various types, such as Montage, LIGO and Cybernetics) in clouds are widespread. How to efficiently schedule them in cloud is very important. This paper formulates the diverse workflows scheduling problem and proposes a CWSA to solve it. The basic idea of CWSA is to select a suitable scheduling strategy for each workflow. Specifically, in CWSA, we design a classification neural network architecture that consists of a graph neural network and a fully connected neural network to classify each workflow to its suitable deadline distribute strategy by its characteristics and deadline constraint. Then CWSA obtains the sub-deadlines of tasks and assigns tasks to appropriate VMs (Virtual Machines). Furthermore, as an important factor in workflow scheduling, the transmission time between dependent tasks is introduced into the graph neural network, which improves the classification accuracy.