Lynn: A Multi-dimensional Dynamic Resource Management System for Distributed Applications in Clouds

Abstract

We present Lynn, a multi-dimensional dynamic resource management system for distributed applications in clouds. Lynn is the first to uniformly manage full kinds of resources, including CPU, memory and particularly, the network I/O bandwidth resource as a whole to schedule tasks. This comprehensive policy allows for more fine-grained control over hardware resources, and thus more efficiency. To support today's sophisticated distributed applications in clouds and to ease the burden of software developers, Lynn defines the concept of virtual cluster in this case and provides a transparent and user-friendly interface of deploying the virtual cluster to run applications. Distributed Applications do not need to make any change to be adapted to Lynn for use. In addition, Lynn puts forward a mechanism that allows different kinds of virtualization techniques to coexist within one single machine, particularly the light-weighted visualization technology Linux container LXC and the famous full virtualization technology KVM. Lynn can handle the difference of the underlying virtualization technology and can manage their resources properly. Then, Lynn breaks the usual rule of static resource allocation of virtual machines. Creatively, Lynn takes careful account of the resources used by each virtual cluster and dynamically adapts their amounts of resources according to their respective workloads. Dynamic policy improves cluster resource utilizations and provides more room for new applications. Our results show that Lynn's multidimensional dynamic resource management policy is feasible and effective.