A Cost-Efficient Resource Provisioning and Scheduling Approach for Deadline-Sensitive MapReduce Computations in Cloud Environment

Abstract

The use of cloud services to process a large amount of data is growing and demands for scalable, reliable, and highly available services in cloud environments are raising. The demands and the urge for developing these facilities have made parallel computing more appealing. Cloud providers offer various types of Virtual Machines (VMs) that are compatible with parallel processing and the clients should pay for their hourly usage. The price varies based on the type, the number and the hiring time of the VMs. A daily price fluctuation timetable has been proposed and scaling the number of VMs on that helps to schedule the computations to meet both deadline and cost minimization goals. It becomes critical to select appropriate VMs and distribute workload efficiently across them. Therefore, the computations and the VMs require being well-managed, scheduled and monitored to meet the deadline while minimizing the total hiring cost. To address these concerns, this paper formulated the problem to calculate the total hiring cost before and during the computations. The execution time and the total cost are calculated based on the application's input size and the required type and the number of VMs. We worked on two applications as sample benchmarks to identify the best approach to choose and manage the VMs to compute them. Our results indicate that among different available approaches for hiring VMs, identifying the most affordable approach leads to minimizing the cost signiflcantly.