Performance evaluation of fair and capacity scheduling in Hadoop YARN

Abstract

Big Data research can be divided broadly into the scheduling of jobs and controlling the rate at which jobs are generating and running. Hadoop YARN provides better resource management schemes to schedule jobs by having a focus on the reduction of total time required to complete the jobs. This paper provides a study of scheduling algorithms in Hadoop YARN and evaluates the performance of two scheduling algorithm, fair scheduling and capacity scheduling using Yarn Scheduler Load Simulator (SLS). The result of this evaluation can be used further to enhance the capabilities of scheduling algorithm in different type of data sets.