Empirical performance evaluation of schedulers for cluster of workstations

Abstract

Cluster computing is receiving exponential popularity as a choice for high performance computing. This is mainly due to its effective cost performance ratio. Resource management systems (RMS) are the key component to manage the resources of clusters efficiently and have a very vital role in the performance of distributed parallel systems especially a job scheduling module. In this paper, we have empirically evaluated four resource management systems (SGE, TORQUE, and MAUI Scheduler and SLURM) with special focus on job scheduler component. These schedulers have been evaluated on a more comprehensive set of metrics such as throughput, CPU, memory and network utilization. Experiments were carried out on three different size testbeds with a range of scheduler configurations such as FCFS, Backfilling, Fair share and SJF scheduling techniques. A head-to-head comparison of different scheduling techniques has also been presented which highlights the effect of RMS on the performance of scheduling techniques. It has been observed from results that relative difference among the performance of scheduling techniques reached up to 63%. We conclude from the experiments that there is no single choice of RMS which can be identified as the best but SLURM performs better than others in most of the cases.