GA-ETI: An enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments

Abstract

Listen

Translate article

Over recent years, cloud computing has become one of the main sources of computer power to run scientific experiments. To cope with these demands, cloud providers need to efficiently match applications with computing resources to maintain an acceptable level of customer satisfaction. A correct match or scheduling of scientific workflows relies on the ability to fully analyze applications prior to execution, analyze characteristics of available computing resources, provide users with several scheduling configurations, and guide users to select the optimal configuration to execute workflows. To date, different schedulers have been proposed to execute complex applications on cloud environments; nevertheless, none exists, to the best of our knowledge, to provide all the aforementioned features. GA-ETI, the scheduler proposed in this work, is designed to address all aforementioned concerns by providing several efficient solutions (in a Pareto Front fashion) to run scientific workflows on cloud resources. Flexibility of optimization procedure of GA-ETI allows it to easily adapt to different types of scientific workflows and produce schedules that effectively exploit/consider the relationship between jobs and their required data. GA-ETI acts as an interface between cloud user and cloud provider in receiving an application, analyzing it, and distributing its tasks among selected resources. GA-ETI differs from the majority of proposed schedulers because it can adapt to the size of both jobs and virtual machines, it includes a monetary cost model (from a public cloud), and it considers complex interdependencies among tasks. We test GA-ETI with five well-known benchmarks with different computing and data transfer demands in our VMware-vSphere private cloud. Through experimentation, GA-ETI has been proved to reduce makespan of executing workflows between 11% and 85% when compared to three up-do-date scheduling algorithms without increasing the monetary cost. GA-ETI opens the way to develop a top-layer-scheduler for a workflow manager system to provide a complex analysis and include different optimizing objectives.