Auto-scaling Solutions Research Articles

Auto-Scaling Techniques in Cloud Computing: Issues and Research Directions.

In the dynamic world of cloud computing, auto-scaling stands as a beacon of efficiency, dynamically aligning resources with fluctuating demands. This paper presents a comprehensive review of auto-scaling techniques, highlighting significant advancements and persisting challenges in the field. First, we overview the fundamental principles and mechanisms of auto-scaling, including its role in improving cost efficiency, performance, and energy consumption in cloud services. We then discuss various strategies employed in auto-scaling, ranging from threshold-based rules and queuing theory to sophisticated machine learning and time series analysis approaches. After that, we explore the critical issues in auto-scaling practices and review several studies that demonstrate how these challenges can be addressed. We then conclude by offering insights into several promising research directions, emphasizing the development of predictive scaling mechanisms and the integration of advanced machine learning techniques to achieve more effective and efficient auto-scaling solutions.

SLA-Adaptive Threshold Adjustment for a Kubernetes Horizontal Pod Autoscaler

Kubernetes is an open-source container orchestration system that provides a built-in module for dynamic resource provisioning named the Horizontal Pod Autoscaler (HPA). The HPA identifies the number of resources to be provisioned by calculating the ratio between the current and target utilisation metrics. The target utilisation metric, or threshold, directly impacts how many and how quickly resources will be provisioned. However, the determination of the threshold that would allow satisfying performance-based Service Level Objectives (SLOs) is a long, error-prone, manual process because it is based on the static threshold principle and requires manual configuration. This might result in underprovisioning or overprovisioning, leading to the inadequate allocation of computing resources or SLO violations. Numerous autoscaling solutions have been introduced as alternatives to the HPA to simplify the process. However, the HPA is still the most widely used solution due to its ease of setup, operation, and seamless integration with other Kubernetes functionalities. The present study proposes a method that utilises exploratory data analysis techniques along with moving average smoothing to identify the target utilisation threshold for the HPA. The objective is to ensure that the system functions without exceeding the maximum number of events that result in a violation of the response time defined in the SLO. A prototype was created to adjust the threshold values dynamically, utilising the proposed method. This prototype enables the evaluation and comparison of the proposed method with the HPA, which has the highest threshold set that meets the performance-based SLOs. The results of the experiments proved that the suggested method adjusts the thresholds to the desired service level with a 1–2% accuracy rate and only 4–10% resource overprovisioning, depending on the type of workload.

Automatic data featurization for enhanced proactive service auto-scaling: Boosting forecasting accuracy and mitigating oscillation

Edge computing has gained widespread adoption for time-sensitive applications by offloading a portion of IoT system workloads from the cloud to edge nodes. However, the limited resources of IoT edge devices hinder service deployment, making auto-scaling crucial for improving resource utilization in response to dynamic workloads. Recent solutions aim to make auto-scaling proactive by predicting future workloads and overcoming the limitations of reactive approaches. These proactive solutions often rely on time-series data analysis and machine learning techniques, especially Long Short-Term Memory (LSTM), thanks to its accuracy and prediction speed. However, existing auto-scaling solutions often suffer from oscillation issues, even when using a cooling-down strategy. Consequently, the efficiency of proactive auto-scaling depends on the prediction model accuracy and the degree of oscillation in the scaling actions.This paper proposes a novel approach to improve prediction accuracy and deal with oscillation issues. Our approach involves an automatic featurization phase that extracts features from time-series workload data, improving the prediction’s accuracy. These extracted features also serve as a grid for controlling oscillation in generated scaling actions. Our experimental results demonstrate the effectiveness of our approach in improving prediction accuracy, mitigating oscillation phenomena, and enhancing the overall auto-scaling performance.

Optimized resource usage with hybrid auto-scaling system for knative serverless edge computing

In the most popular serverless platform - Knative, dynamic resource allocation is implemented using horizontal auto-scaling algorithms to create or delete service instances based on different monitored metrics. However, the assigned resources for each instance are fixed. Vertical scaling up or down assigned resources per instance is required to avoid over-provisioning resources which are limited at the edge. Hybrid (horizontal and vertical) auto-scaling solutions proposed by existing works have several limitations. These solutions are optimized for separated services and get degraded performance when applied in a normal environment with multiple concurrent services. Further, most methods make significant changes to the original Knative platform, and have not been considered to be adopted since then. In this article, instead of Knative modification, we developed separated Kubernetes operators and custom resources (CRs) that can assist the Knative auto-scaler with optimal hybrid auto-scaling configurations based on traffic prediction. First, we characterize each service with a profile of different assigned resource levels pairing with their optimal target Knative’s horizontal scaling request concurrency. Then, based on these profiles, we calculate the best-assigned resources level, target concurrency level, and the number of required instances corresponding to each future time step’s predicted traffic. Finally, these configurations are applied to Knative’s default auto-scaler and services’ CR. Experiments done on our testbed compared our solution with a Knative hybrid auto-scaler solution that does not consider the service’s target request concurrency, and the default Knative horizontal auto-scaler. The results show our solution improvements up to 14% and 20% in terms of resource usage, respectively.

Hybrid Autoscaling Strategy on Container-Based Cloud Platform

The container has several advantages over the traditional virtual machine technology such as light-weight, fast booting time, and fast recovery. Kubernetes is one the most outstanding container management and deployment platforms. The Kubernetes provides autoscaling function, which will increase and decrease the hardware resources to adapt with the current traffic load situation to keep the user experience. Two popular autoscaling methods are horizontal autoscaling and vertical autoscaling. Based on the monitoring resource utilization, horizontal autoscaling will increase the number of PoDs (point of deployment) or vertical autoscaling will increase the hardware resources of each PoD to achieve the target utilization. In this paper, we present a hybrid solution that combines the advantages of both autoscaling solutions and proposes a bandwidth-efficient scheduler strategy. By numerical analysis, our hybrid approach is better than the normal HPA approach in terms of bandwidth cost and has lower autoscaling latency than the VPA approach

Web Application Resource Requirements Estimation Based on the Workload Latent Features

Most cloud computing platforms offer reactive resource auto-scaling mechanisms for dealing with variable traffic patterns to deliver the desired QoS properties while keeping low provisioning costs. However, a range of scenarios have not been fully addressed by the current auto-scaling solutions, particularly dealing with a rapid increase in workload and the risk of thrashing due to frequent workload variations. A reactive system is vulnerable in such conditions. Realizing the full potential of auto-scaling still remains challenging particularly due to the need of accurately estimating the application resource requirements for time-varying workload patterns. In this work, we propose and evaluate a novel method using only application access logs to estimate more accurately the hardware resource demands and application response time. In particular, we propose novel workload latent features which we compute by applying unsupervised learning on the access logs. We use these latent features to estimate the application hardware resource requirements and response time for various workload patterns. We evaluate the proposed method using multiple benchmark web applications and compare it with current state-of-the-art. Extensive experimental evaluations show an excellent performance of our proposed workload latent features in estimating response time, CPU, memory, and bandwidth utilization.

Design and evaluation of a biologically-inspired cloud elasticity framework

The elasticity in cloud is essential to the effective management of computational resources as it enables readjustment at runtime to meet application demands. Over the years, researchers and practitioners have proposed many auto-scaling solutions using versatile techniques ranging from simple if-then-else based rules to sophisticated optimisation, control theory and machine learning based methods. However, despite an extensive range of existing elasticity research, the aim of implementing an efficient scaling technique that satisfies the actual demands is still a challenge to achieve. The existing methods suffer from issues like: (1) the lack of adaptability and static scaling behaviour whilst considering completely fixed approaches; (2) the burden of additional computational overhead, the inability to cope with the sudden changes in the workload behaviour and the preference of adaptability over reliability at runtime whilst considering the fully dynamic approaches; and (3) the lack of considering uncertainty aspects while designing auto-scaling solutions. In this paper, we aim to address these issues using a holistic biologically-inspired feedback switch controller. This method utilises multiple controllers and a switching mechanism, implemented using fuzzy system, that realises the selection of suitable controller at runtime. The fuzzy system also facilitates the design of qualitative elasticity rules. Furthermore, to improve the possibility of avoiding the oscillatory behaviour (a problem commonly associated with switch methodologies), this paper integrates a biologically-inspired computational model of action selection. Lastly, we identify seven different kinds of real workload patterns and utilise them to evaluate the performance of the proposed method against the state-of-the-art approaches. The obtained computational results demonstrate that the proposed method results in achieving better performance without incurring any additional cost in comparison to the state-of-the-art approaches.

Adaptive Microservice Scaling for Elastic Applications

Today, Internet users expect Web applications to be fast, performant, and always available. With the emergence of Internet of Things (IoT), data collection and the analysis of streams have become more and more challenging. Behind the scenes, application owners and cloud service providers work to meet these expectations, yet, the problem of how to most effectively and efficiently auto-scale a Web application to optimize for performance while reducing costs and energy usage is still a challenge. In particular, this problem has new relevance due to the continued rise of IoT and microservice-based architectures. A key concern, that is often not addressed by current auto-scaling systems, is the decision on which microservice to scale in order to increase performance. Our aim is to design a prototype auto-scaling system for microservice-based Web applications that can learn from the past service experience. The contributions of the work can be divided into two parts: 1) developing a pipeline for microservice auto-scaling and 2) evaluating a hybrid sequence and supervised learning model for recommending scaling actions. The pipeline has proven to be an effective platform for exploring auto-scaling solutions, as we will demonstrate through the evaluation of our proposed hybrid model. The results of the hybrid model show the merit of using a supervised model to identify which microservices should be scaled up more.

Multilayered Autoscaling Performance Evaluation: Can Virtual Machines and Containers Co–Scale?

Abstract The wide adoption of cloud computing by businesses is due to several reasons, among which the elasticity of the cloud virtual infrastructure is the definite leader. Container technology allows increasing the flexibility of an application by adding another layer of virtualization. The containers can be dynamically created and terminated, and also moved from one host to another. A company can achieve a significant cost reduction and increase the manageability of its applications by allowing the running of containerized microservice applications in the cloud. Scaling for such solutions is conducted on both the virtual infrastructure layer and the container layer. Scaling on both layers needs to be synchronized so that, for example, the virtual machine is not terminated with containers still running on it. The synchronization between layers is enabled by multilayered cooperative scaling, implying that the autoscaling solution of the virtual infrastructure layers is aware of the decisions of the autoscaling solution on the container layer and vice versa. In this paper, we introduce the notion of cooperative multilayered scaling and the performance of multilayered autoscaling solutions evaluated using the approach implemented in ScaleX (previously known as Autoscaling Performance Measurement Tool, APMT). We provide the results of the experimental evaluation of multilayered autoscaling performance for the combination of virtual infrastructure autoscaling of AWS, Microsoft Azure and Google Compute Engine with pods horizontal autoscaling of Kubernetes by using ScaleX with four distinct load patterns. We also discuss the effect of the Docker container image size and its pulling policy on the scaling performance.

The Effect of the Resource Consumption Characteristics of Cloud Applications on the Efficiency of Low-Metric Auto Scaling Solutions

The Effect of the Resource Consumption Characteristics of Cloud Applications on the Efficiency of Low-Metric Auto Scaling Solutions

DEPAS: a decentralized probabilistic algorithm for auto-scaling

The dynamic provisioning of virtualized resources offered by cloud computing infrastructures allows applications deployed in a cloud environment to automatically increase and decrease the amount of used resources. This capability is called auto-scaling and its main purpose is to automatically adjust the scale of the system that is running the application to satisfy the varying workload with minimum resource utilization. The need for auto-scaling is particularly important during workload peaks, in which applications may need to scale up to extremely large-scale systems. Both the research community and the main cloud providers have already developed auto-scaling solutions. However, most research solutions are centralized and not suitable for managing large-scale systems, moreover cloud providers’ solutions are bound to the limitations of a specific provider in terms of resource prices, availability, reliability, and connectivity. In this paper we propose DEPAS, a decentralized probabilistic auto-scaling algorithm integrated into a P2P architecture that is cloud provider independent, thus allowing the auto-scaling of services over multiple cloud infrastructures at the same time. Our experiments (simulations and real deployments), which are based on real service traces, show that our approach is capable of: (i) keeping the overall utilization of all the instantiated cloud resources in a target range, (ii) maintaining service response times close to the ones obtained using optimal centralized auto-scaling approaches.