YISHAN: Managing Large-scale Cloud Database Instances via Machine Learning

Abstract

Efficiently managing database instances over cloud-scale clusters is significant for increasing service quality and reducing operational cost, especially confronting the growing cluster size and heterogeneous application services. Alibaba Cloud provides a large-scale Relational Database Service (RDS) for millions of users including enterprises from start-ups to large international corporations. To manage tremendous amount of RDS instances in the Cloud with the goal of reducing cost while guaranteeing service level agreement(SLA), YISHAN, an intelligent database instance management system, is designed to dynamically manage the placement of instances using machine learning techniques. YISHAN collects historical performance data to analyze patterns of the resource utilization of instances and hosts. By learning “good packings” in which instances colocate harmoniously, YISHAN is able to optimize the instances placement to provide better quality of service and improve the efficiency of CPU, memory, and disk resources. We deploy and run YISHAN in Alibaba Cloud RDS. The running logs show that YISHAN successfully saves 17% of the resources in hosts and efficiently reduces the burdens and crash risks of RDS instances.