Multi-resource Fair Sharing for Multiclass Workflows

Abstract

Multi-resource sharing for concurrent workows necessitates a fairness criteria to allocate multiple resources to work-flows with heterogeneous demands. Recently, this problem has attracted increasing attention and has been investigated by assuming that each workow has a single class of jobs and that each class contains jobs of the same demand profile. The demand profile of a class represents the required multi-resources of a job. However, for typical applications in cloud computing and distributed data processing systems, a workow usually needs to process multiple classes of jobs. Relying on the concept of slowdown, we characterize fairness for multi-resource sharing and address scheduling for multiclass workows. We optimize the mixture of different classes of jobs for a workow as optimal operation points to achieve the least slowdown, and discuss desirable properties for these operation points. These studies assume that the jobs are infinitely divisible. When jobs are non-preemptive and indivisible, any fairness criteria that only relies on the instantaneous resource allocation cannot be strictly maintained at every time point. To this end, we relax the instantaneous fairness to an average metric within a time interval. This relaxation introduces a time average to fairness and allows occasional, but not too often, violations of instantaneous fairness. In addition, it brings exibility and opportunities for further optimization on resource utilization, e.g., using bin-packing, within the constraint on fairness.