Performance Related Tradeoffs between Fairness and Throughput for Job Scheduling in Cloud Environment

Abstract

Listen

Translate article

The burst of data volume and application complexity, it has become prevalent to host large-scale computations in clusters of distributed servers. In shared production clusters, job scheduling is of paramount importance to the cluster performance. The two basic scheduling objectives are efficiency and fairness---an ideal scheduler shall facilitate fast job response, and meanwhile avoid starvation by guaranteeing worst-case service quality to each job.For inter-job scheduling, efficiency and fairness are conflicting with each other, leading to a dilemma of either predictable performance at the expense of long response time, or minimum mean response time at the risk of starvation. As a result, it’s critical to develop resource scheduling strategies that can do well in both worlds. In this regard, we make the following contributions.The study of existing scheduling approaches is carried out and two new approaches have been proposed. In the first approach- ‘Novel Hybrid Cost based Priority (NHCP)’, the tasks are assigned profit threshold and priorities. In the second approach- ‘Two Party Model (TPM)’, that allows lower priority tasks to be migrated and allocated from slow to fast resource even during low priority if fast resource is free i.e. higher priority resource. In both the approaches we have maximized throughput, minimized makespan and also tried to reduce average waiting time. It is observed that proposed approaches perform better than the existing approaches under different work load conditions.