Microscaler: Cost-Effective Scaling for Microservice Applications in the Cloud With an Online Learning Approach

Abstract

Recently, the microservice becomes a popular architecture to construct cloud native systems due to its agility. In cloud native systems, autoscaling is a key enabling technique to adapt to workload changes by acquiring or releasing the right amount of computing resources. However, it becomes a challenging problem in microservice applications, since such an application usually comprises a large number of different microservices with complex interactions. When the performance decreases due to an unpredictable workload peak, it is difficult to pinpoint the scaling-needed services which need to scale out and evaluate how many resources they need. In this article, we present a novel system named <i>Microscaler</i> to automatically identify the scaling-needed services and scale them to meet the Service Level Agreement (SLA) with an optimal cost for microservice applications. <i>Microscaler</i> first collects the quality of service (QoS) metrics in the service mesh enabled microservice infrastructure. Then, it determines under-provisioning or over-provisioning service instances along the service dependency graph with a novel scaling-needed service criterion named <i>service power</i>. The service dependency graph could be obtained by correlating each request flow in the service mesh. By combining an online learning approach and a step-by-step heuristic approach, <i>Microscaler</i> can precisely reach the optimal service scale meeting the SLA requirements. The experimental evaluations in a microservice benchmark show that <i>Microscaler</i> achieves an average 93 percent precision in scaling-needed service determination and converges to the optimal service scale faster than several state-of-the-art methods. Moreover, <i>Microscaler</i> is lightweight and flexible enough to work in a large-scale microservice system.