Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments

Abstract

In recent years, cloud platforms have been rapidly developed and deployed around the globe and many large-scale scientific workflows have been migrated to multiple clouds for cost-effective data analysis. In such cloud-based workflow applications, financial cost is a major concern in addition to traditional performance requirements such as execution time. In this paper, we formulate a workflow mapping problem to minimize the financial cost of deadline-constrained scientific workflows executed in multi-cloud environments, referred to as MinCost-MC, which is shown to be NP-complete. Within a generic three-layer workflow execution framework, we propose a Workflow Mapping algorithm for Financial Cost Optimization, referred to as WMFCO. This algorithm takes in consideration storage requirements, I/O operations, and data transfers to minimize the financial cost of a given workflow within a specified deadline. Extensive simulation results show that WMFCO exhibits a superior performance over existing algorithms in terms of financial cost in multi-cloud environments.