Multitasking bi-level evolutionary algorithm for data-intensive scientific workflows on clouds

Abstract

With the deployment of workflow and other applications, cloud computing is accessible and offers assistance for optimizing workflow execution and enhancing performance. Existing research, however, tends to disregard the influence of dataset migration on workflow execution and focuses more on task execution time. This study suggests a new model for the problem of data-intensive workflow execution. Firstly, according to the structure of the workflow scheduling problem, it is divided into two sub-problems: data placement and task scheduling. The two sub-problems interact with each other and a bi-level optimum model is established. By seeking a better allocation strategy for the dataset placement and then seeking the best task-scheduling solution. Secondly, an improved multitasking bi-level evolutionary algorithm (IM-BLEA) is proposed. When dealing with the lower-level optimization problem (LLOP), offspring are selected by sorting individuals by their performance and overall performance in the population, and this environmental selection enhances the diversity and searchability of the population. Finally, compared with the other multitasking algorithm, IM-BLEA has good performance. Simulation results based on real scientific workflows show that the algorithm improves the values of transfer time and number of selected data centers by 56% and 10% compared to the comparison algorithm.