Improving reliability and reducing cost of task execution on preemptible VM instances using machine learning approach

Abstract

Cloud users can acquire resources in the form of virtual machines (VMs) instances for computing. These instances can be on-demand, reserved and spot instances. Spot-priced virtual machines are offered at the reduced cost compared to on-demand and reserved but are unreliable to use as their availability depends on user’s bid. To use spot instances (preemptible VMs), users have to bid for resources and trade-off between monetary cost and reliability as reliability increases with the increase in cost of execution. The cost of execution can be reduced significantly with the use of preemptible VM instances. These instances are only available until users bid higher in comparison with spot price that is fixed by the cloud providers. Hence, it becomes a critical challenge to minimize the associated cost and increases the reliability for a given deadline. In this article, an algorithm has been designed for predicting the spot price to facilitate the users in bidding. Further, a checkpointing algorithm has been proposed for saving the task’s progress at optimal time intervals by the use of the proposed spot price prediction algorithm. The proposed algorithms in the article emphasize the use of preprocessed data for prediction of prices in short intervals. The prediction algorithm is based on machine learning techniques. It predicts the price and provides a comprehensive comparison for prediction of the prices for long term (12 h) as well as short term (10 min). For predicting the long-term and short-term prices, different machine learning techniques have been used on the basis of least error in prediction. The best suitable machine learning algorithm with least error is selected for prediction as well as checkpointing. Using these algorithms, one can improve reliability and reduce cost of computing on preemptible VM instances significantly. To the best of our knowledge, this is the first attempt of its kind in this field.