A survey on job scheduling algorithms in Big data processing

Abstract

Scheduling problem has been an active area of research in computing systems since their inception. The Apache Hadoop framework has emerged as most widely adopted framework for distributed data processing because of open source and allowing use of commodity hardware. Job scheduling has become an important factor to achieve high performance in Hadoop cluster. Several scheduling algorithms have been developed for Hadoop-MapReduce model which vary widely in design and behavior, handling different issues such as locality of data, user share fairness and resource awareness. This paper highlights fundamental issues in job scheduling, presents classification of Hadoop schedulers, and discusses presented survey of existing scheduling algorithm. Moreover paper also discusses features, advantages, and limitations of the scheduling algorithms. This paper also discusses about how various resource monitoring tools or frameworks help in achieving better result from MapReduce. It also discusses customized MapReduce frameworks used for improving the performance. This paper would be useful to beginners and researchers for understanding the state-of-the-art on scheduling in Big data processing.