Job scheduling for big data analytical applications in clouds: A taxonomy study

Abstract

Cloud environments have provided great help for the development of big data. Today, cloud environments have become the most active platform for big data analysis applications (BDAAs). Cloud products provide convenience for BDAA providers to provision big data analysis services to their users at a more cost-effective way. Rational utilization of cloud resources can bring higher efficiency and lower cost to BDAAs. Therefore, how to better process big data jobs in cloud environments has become a hot research topic in recent years. Efficient job scheduling schemes can make full use of cloud resources, and then achieve different goals, such as saving energy, minimizing cost and maximizing job execution efficiency. So designing suitable scheduling schemes for different problem scenarios has become the most common and effective method to optimize the execution of big data analysis jobs. To explore practical and efficient job scheduling algorithms in different scenarios, references of existing scheduling algorithms are often needed. Therefore, it is necessary to survey the research work of BDAA job scheduling in cloud environments in recent years, which can provide detailed guidance and help for future research in this field. This paper summarizes the work related to BDAA job scheduling in cloud environments, selects articles related to the topic of this survey from conferences and journals from 2016 to 2021, and divides scheduling algorithms with different research focuses into several categories. On this basis, the advantages and disadvantages of existing research work are compared, and the directions and challenges for future research are pointed out.