Proactive Random-Forest Autoscaler for Microservice Resource Allocation

Abstract

Cloud service providers have been shifting their workloads to microservices to take advantage of their modularity, flexibility, agility, and scalability. However, numerous obstacles remain to achieving the most out of microservice deployments, especially in terms of a Quality of Service (QoS). One possible approach to overcoming these obstacles is to perform autoscaling, which is the ability of cloud infrastructure and services to scale themselves up or down by changing their resource pool. There are two major categories of autoscaling: reactive and proactive. In reactive autoscaling, a feedback loop based on current workload resource usage is implemented to guide resource scaling. One disadvantage of reactive autoscaling is that it may result in inconsistencies between workload demand and resource allocation. In proactive autoscaling, a prediction model is used to guide the future allocation of resources according to current workload metrics. In this paper, a novel proactive autoscaling method is introduced where a two-state, machine-learning Random Forest (RF) model is designed to forecast the future CPU and memory utilization values required by the microservice workload. These predicted values are then used to adjust the resource pool both vertically (hardware resources) and horizontally (microservice replicas). The RF proactive autoscaler has been implemented on a home-grown, open-source microservice prototyping platform and verified using real-world workloads. The experiments show that the RF proactive autoscaler outperforms state-of-the-art ones in terms of allocated resources and latency. The increase in the utilization of allocated resources can reach 90% and the improvement in end-to-end latency, measured by the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$95^{th}$ </tex-math></inline-formula> percentile, can reach 95%.