A Cooperative Coevolution Hyper-Heuristic Framework for Workflow Scheduling Problem

Abstract

Workflow scheduling problem (WSP) is a well-known combinatorial optimization problem, which is defined to assign a series of interconnected tasks to the available resources to meet user defined Quality of Service (QoS). The guided random search methods and heuristic based methods are two most common methods for solving WSP. However, these methods either require expensive computational cost or heavily rely on human's empirical knowledge, which makes them inconvenient for practical applications. Keeping this in mind, this paper proposes a cooperative coevolution hyper-heuristic framework to solve WSP with an objective of minimizing the completed time of workflow. In particular, in the proposed framework, two heuristic rules, namely, the task selection rule (TSR) and the resource selection rule (RSR), are learned automatically by a cooperative coevolution genetic programming (CCGP) algorithm. The TSR is used to select a ready task for scheduling, while the RSR is used to allocate resources to perform the selected task. To improve the search efficiency, a set of low-level heuristics are defined and used as building blocks to construct the TSR and RSR. Further, to validate the effectiveness of the proposed framework, randomly generated workflow instances and four real-world workflows are used as test cases in the experimental study. Compared with several state-of-the-art methods, e.g., the Heterogeneous Earliest Finish Time (HEFT) and the Predict Earliest Finish Time (PEFT), the high-level heuristics found by our proposed framework demonstrate superior performance on all the test cases in terms of several metrics including the schedule length ratio, speedup and efficiency.