Multi-resource schedulable unit for adaptive application-driven unified resource management in data centers

Abstract

Applications in modern data centers have a wide variety of resource requirements along the four main dimensions of computing, memory, storage, and networking. Data centers must manage these resources separately for each dimension, resulting in highly inefficient allocation of precious resources or even disastrous schemes that contribute to low utilization or over-provisioning of resources. However, concerted efforts to jointly optimize all types of resources in the same framework appears insurmountable due to an exponentially increasing complexity linked to the thousands of fine-grained resources such as CPUs, memory blocks, disk blocks, switches, etc., which results in an astronomically large number of possible resource combinations. In this paper, we present a novel multi-resource scheduling approach that keeps the complexity to a minimum while it provides efficient resource utilization by keeping a sufficient level of granularity. Our scheme is based on the idea of defining a new finest-grain schedulable unit called multi-resource schedulable unit (MRSU) that will be used by the scheduler to allocate resources to applications in the data center. The problem of MRSU-based scheduling is modeled with an optimization problem and several solutions are proposed. Our results show a significant performance improvement of our technique over conventional static schedulers based on virtual machines (VMs) in terms of saved over-allocation and application satisfaction.