Elastic Scaling Research Articles

Edge computing resource scheduling method based on container elastic scaling.

Edge computing is a crucial technology to solve the problem of computing resources and bandwidth required for extensive edge data processing, as well as for meeting the real-time demands of applications. Container virtualization technology has become the underlying technical basis for edge computing due to its efficient performance. Because the traditional container scaling strategy has issues such as long response times, low resource utilization, and unpredictable container application loads, this article proposes a method for scheduling edge computing resources based on the elastic scaling of containers. Firstly, a container load prediction model (Trend Enhanced-Temporal Convolutional Network, TE-TCN) is designed based on the temporal convolutional neural network, which features an encoder-decoder structure. The encoder extracts potential temporal relationship features from the historical data of the container load, while the decoder identifies the trend item of the container load through the trend enhancement module. Subsequently, the information extracted by the encoder and decoder is fed into the fully connected layer to facilitate container load prediction using the dual-input ResNet method. Secondly, Markov decision process (MDP) is used to model the elastic expansion problem of containers in multi-objective optimization. Utilizing the prediction outcomes of the TE-TCN load prediction model, a time-varying action space is formulated to address the issue of excessive action space in conventional reinforcement learning. Subsequently, a predictive container scaling strategy based on reinforcement learning is devised to align with the application load patterns in the container environment, enabling adaptation to the surge in traffic generated by the container environment. Finally, the experimental results on the WorldCup98 dataset and the real dataset show that the TE-TCN model can accurately predict the container load change. Experiments in the actual environment demonstrate that the proposed strategy reduces the average response time by 16.2% when the burst load arrives, and increases the average CPU utilization by 44.6% when the jitter load occurs.

Analysis of Aliyun-based serverless on MapReduce efficiency

In the context of the current era of big data, traditional Hadoop and cluster-based MapReduce frameworks are unable to meet the demands of modern research. This paper presents a MapReduce framework based on the AliCloud Serverless platform, which has been developed with the objective of optimizing word frequency counting in large-scale English texts. Leveraging AliCloud's dynamic resource allocation and elastic scaling, we have created an efficient and flexible text data processing system. This paper details the design and implementation of the Map and Reduce phases and analyses the impact of vCPU and memory specifications, as well as parallel resource allocation on system performance. Experimental results show that increasing vCPU specifications significantly improves processing capacity and execution efficiency. While the impact of memory specifications is relatively minor, it can positively influence performance in specific scenarios. Parallel processing markedly enhances system performance. Experiments on "Harry Potter and the Sorcerer's Stone" validate the framework's performance across various configurations. This study offers valuable insights for the design and optimization of serverless-based MapReduce frameworks, as well as suggesting future enhancements. These include the implementation of advanced parallel computing strategies, improved error handling, and refined data preprocessing, which collectively aim to boost system performance and stability.

Materials data science using CRADLE: A distributed, data-centric approach

There is a paradigm shift towards data-centric AI, where model efficacy relies on quality, unified data. The common research analytics and data lifecycle environment (CRADLE™) is an infrastructure and framework that supports a data-centric paradigm and materials data science at scale through heterogeneous data management, elastic scaling, and accessible interfaces. We demonstrate CRADLE’s capabilities through five materials science studies: phase identification in X-ray diffraction, defect segmentation in X-ray computed tomography, polymer crystallization analysis in atomic force microscopy, feature extraction from additive manufacturing, and geospatial data fusion. CRADLE catalyzes scalable, reproducible insights to transform how data is captured, stored, and analyzed.Graphical abstract

Orchestrating scheduling, grouping and parallelism to enhance the performance of distributed stream computing system

In a big data stream computing environment, the arrival rate of data streams usually fluctuates over time, posing a great challenge to the elasticity of system. The performance of stream computing system is crucial, especially when dealing with unbounded and fluctuating data streams. Most prior studies have primarily focused on one or two aspects to enable elasticity, often lacking prompt and comprehensive performance optimization. This limitation could lead to a tuning bottleneck, preventing the system’s performance from consistently reaching its optimal state. Additionally, many stream computing systems are not intelligently adaptive in real time due to the challenges of manual parameter reconfiguration for fluctuating streams. To better address these issues, we propose a framework named Sgp-Stream, which orchestrates scheduling, grouping and parallelism (Sgp). To enhance the system performance. We conduct the following research: (1) Running experiments to evaluate the impact of different factors such as scheduling, grouping and parallelism on system performance. Results show that factors at a single level usually have an upper limit on tuning system performance, and better overall performance can be achieved by coordinating multi-level factors. (2) Establishing quantitative models for stream application that consider computational cost and communication cost, multi-dimensional featured data stream, data center resources, and latency & throughput performance. (3) Demonstrating the effectiveness of the proposed runtime-aware data stream grouping based on smooth weighted polling, elastic adaptive scheduling based on Linear Deterministic Greedy and elastic scaling strategy based on Gradient Descent in Sgp-Stream, for continuous performance optimization.(4) Evaluating the application latency, throughput and resource utilization objectives using a real-world elastic stream computing system and twitter data set. Experimental results show that, compared to existing state-of-the-art works, the proposed Sgp-Stream outperforms them by reducing latency by 26%–48%, improving throughput by 14%–20%, and increasing resource utilization rate by 15%–21%, especially under increasing data stream input rates.

Scalable Distributed Inverted List Indexes in Disaggregated Memory

Memory disaggregation separates compute (CPU) and main memory resources into disjoint physical units to enable elastic and independent scaling. Connected via high-speed RDMA-enabled networks, compute nodes can directly access remote memory. This setting often requires complex protocols with many network roundtrips as memory nodes have near-zero compute power. In this paper, we design a scalable distributed inverted list index for disaggregated memory architectures. An inverted list index maps a set of terms to lists of documents that contain this term. Current solutions either partition the index horizontally or vertically with severe limitations in the disaggregated memory setting due to data and access skew, high network latency, or out-of-memory errors. Our method partitions lists into fixed-size blocks and spreads them across the memory nodes to balance skewed accesses. Block-based list processing keeps the memory footprint of compute nodes low and masks latency by interleaving remote accesses with expensive list operations. In addition, we propose efficient updates with optimistic concurrency control and read-write conflict detection. Our experiments confirm the efficiency and scalability of our method.

Understanding the Performance Implications of the Design Principles in Storage-Disaggregated Databases

Storage-compute disaggregation has recently emerged as a novel architecture in modern data centers, particularly in the cloud. By decoupling compute from storage, this new architecture enables independent and elastic scaling of compute and storage resources, potentially increasing resource utilization and reducing overall costs. To best leverage the disaggregated architecture, a new breed of database systems termed storage-disaggregated databases has recently been developed, such as Amazon Aurora, Microsoft Socrates, Google AlloyDB, Alibaba PolarDB, and Huawei Taurus. However, little is known about the effectiveness of the design principles in these databases since they are typically developed by industry giants, and only the overall performance results are presented without detailing the impact of individual design principles. As a result, many critical research questions remain unclear, such as the performance impact of storage-disaggregation, the log-as-the-database design, shared-storage, and various log-replay methods. In this paper, we investigate the performance implications of the design principles that are widely adopted in storage-disaggregated databases for the first time. As these databases were usually not open-sourced, we have made a significant effort to implement a storage-disaggregated database prototype based on PostgreSQL v13.0. By fully controlling and instrumenting the codebase, we are able to selectively enable and disable individual optimizations and techniques to evaluate their impact on performance in various scenarios. Furthermore, we open-source our storage-disaggregated database prototype for use by the broader database research community, fostering collaboration and innovation in this field.

Optimization enabled elastic scaling in cloud based on predicted load for resource management

Cloud computing epitomizes an important invention in the field of Information Technology, which presents users with a way of providing on-demand access to a pool of shared computing resources. A major challenge faced by the cloud system is to assign the exact quantity of resources to the users based on the demand, while meeting the Service Level Agreement (SLA). Elasticity is a major aspect that provides the cloud with the capability of adding and removing resources “on the fly” for handling load variations. However, elastic scaling requires suspension of the application tasks forcibly, while performing resource distribution; thereby Quality of Service (QoS) gets affected. In this research, an elastic scaling approach based on optimization is developed which aims at attaining an improved user experience. Here, load prediction is performed based on various factors, like bandwidth, CPU, and memory. Later, horizontal as well as vertical scaling is performed based on the predicted load using the devised leader Harris honey badger algorithm. The devised optimization enabled elastic scaling is evaluated for its effectiveness based on metrics, such as predicted load error, cost, and resource utilization, and is found to have attained values of 0.0193, 153.581, and 0.3217.

Hybrid Elastic Scaling Strategy for Container Cloud based on Load Prediction and Reinforcement Learning

To harness the advantages of both proactive and responsive scaling, adapting to various workload scenarios, this paper introduces a container hybrid scaling strategy called HyPredRL, rooted in load prediction and reinforcement learning. Within the proactive scaling module RL-PM, a load prediction model, MSC-LSTM, predict workloads and, in conjunction with current workload states, leverages reinforcement learning agents for intelligent scaling decisions. The responsive scaling strategy, SLA-HPA, enhances Kubernetes’ native scaling strategy, which primarily considers resource utilization, by incorporating response time metrics. Ultimately, a hybrid scaling controller is designed, applying the principles of “rapid scaling out” and “balanced conflicts” to coordinate proactive and responsive scaling. Experimental results demonstrate that HyPredRL outperforms existing methods in SLA violation rate, resource utilization, and request response time, effectively improving application performance and scalability.

Rcmp: Reconstructing RDMA-Based Memory Disaggregation via CXL

Memory disaggregation is a promising architecture for modern datacenters that separates compute and memory resources into independent pools connected by ultra-fast networks, which can improve memory utilization, reduce cost, and enable elastic scaling of compute and memory resources. However, existing memory disaggregation solutions based on remote direct memory access (RDMA) suffer from high latency and additional overheads including page faults and code refactoring. Emerging cache-coherent interconnects such as CXL offer opportunities to reconstruct high-performance memory disaggregation. However, existing CXL-based approaches have physical distance limitation and cannot be deployed across racks. In this article, we propose Rcmp, a novel low-latency and highly scalable memory disaggregation system based on RDMA and CXL. The significant feature is that Rcmp improves the performance of RDMA-based systems via CXL, and leverages RDMA to overcome CXL’s distance limitation. To address the challenges of the mismatch between RDMA and CXL in terms of granularity, communication, and performance, Rcmp (1) provides a global page-based memory space management and enables fine-grained data access, (2) designs an efficient communication mechanism to avoid communication blocking issues, (3) proposes a hot-page identification and swapping strategy to reduce RDMA communications, and (4) designs an RDMA-optimized RPC framework to accelerate RDMA transfers. We implement a prototype of Rcmp and evaluate its performance by using micro-benchmarks and running a key-value store with YCSB benchmarks. The results show that Rcmp can achieve 5.2× lower latency and 3.8× higher throughput than RDMA-based systems. We also demonstrate that Rcmp can scale well with the increasing number of nodes without compromising performance.

Elasticity unleashed: Fine-grained cloud scaling through distributed three-way decision fusion with multi-head attention

With the proliferation of cloud computing, elastic resource scaling has emerged as a critical challenge. Cloud platforms must efficiently allocate resources to match dynamic workloads. This study addresses the key trade-off between responsiveness and prudence in scaling a novel distributed three-way decision fusion approach. Distributed three-way decisions of immediate, delayed, or no scaling are made at coarse and fine-time granularity levels. Subsequently, a multi-head attention mechanism intelligently fuses these decisions, weighing the relevance of the different granularities to synthesize an integrated scaling strategy. The reactiveness of sudden workload changes are balanced during fusion by considering long-term trends. Comprehensive experiments on real-world data demonstrated that the proposed fusion strategy substantially improves resource efficiency and adherence to service-level agreements compared to existing methods. Multi-head attention imparts autonomy in adapting to diverse operating conditions. The proposed principled methodology and attention-based decision aggregation hold significance for efficient, adaptive, and interpretable cloud scaling mechanisms.

Secure and Scalable Architectures for Cloud Computing

Cloud computing has emerged as a cornerstone of modern computing infrastructure, offering unparalleled scalability and flexibility for diverse applications. However, the widespread adoption of cloud services has also raised significant concerns regarding security and scalability. As organizations increasingly rely on cloud environments to store and process sensitive data, ensuring robust security measures while maintaining scalability becomes paramount. This paper presents a comprehensive examination of secure and scalable architectures for cloud computing, addressing the challenges and proposing innovative solutions to enhance the resilience and efficiency of cloud infrastructures. The first section of the paper delves into the fundamental principles of cloud computing, outlining its key characteristics and architectural components. By understanding the underlying architecture of cloud systems, stakeholders can better appreciate the intricate interplay between security and scalability. Subsequently, the paper elucidates the multifaceted security threats facing cloud environments, ranging from data breaches and malicious attacks to insider threats and compliance issues. These challenges underscore the critical importance of implementing robust security mechanisms to safeguard sensitive data and infrastructure resources. In response to these challenges, the paper presents a systematic analysis of existing security mechanisms and architectural paradigms designed to fortify cloud environments. This includes encryption techniques, access control mechanisms, authentication protocols, and intrusion detection systems, among others. Furthermore, the paper explores the concept of defense-in-depth strategies, emphasizing the importance of layering security measures to mitigate the impact of potential breaches and vulnerabilities. In tandem with security considerations, the paper examines strategies for achieving scalability in cloud architectures. Scalability is a defining characteristic of cloud computing, enabling on-demand resource allocation and elastic scaling to accommodate fluctuating workloads. However, achieving scalability while ensuring security presents unique challenges, particularly in multi-tenant environments where resources are shared among disparate users. To address these challenges, the paper explores architectural frameworks such as microservices, containerization, and serverless computing, which offer inherent scalability benefits while facilitating robust isolation and security. Moreover, the paper investigates the role of automation and orchestration in optimizing cloud scalability and security. Through the use of DevOps practices and infrastructure- as-code tools, organizations can streamline deployment processes, enforce security policies, and dynamically scale resources in response to evolving demands. Additionally, the paper examines emerging technologies such as edge computing and federated architectures, which extend the scalability and security benefits of cloud computing to edge devices and distributed environments. In conclusion, this paper underscores the intrinsic link between security and scalability in cloud computing architectures. By adopting a holistic approach that integrates robust security measures with scalable design principles, organizations can mitigate risks, enhance operational efficiency, and unlock the full potential of cloud computing. As the landscape of cloud technologies continues to evolve, ongoing research and innovation in secure and scalable architectures will remain essential to address emerging threats and optimize performance in cloud environments.

Elastic Scaling of Stateful Operators Over Fluctuating Data Streams

Elastic Scaling of Stateful Operators Over Fluctuating Data Streams

SepHash: A Write-Optimized Hash Index On Disaggregated Memory via Separate Segment Structure

Disaggregated memory separates compute and memory resources into independent pools connected by fast RDMA (Remote Direct Memory Access) networks, which can improve memory utilization, reduce cost, and enable elastic scaling of compute and memory resources. Hash indexes provide high-performance single-point operations and are widely used in distributed systems and databases. However, under disaggregated memory, existing hash indexes suffer from write performance degradation due to high resize overhead and concurrency control overhead. Traditional write-optimized hash indexes are not efficient for disaggregated memory and sacrifice read performance. In this paper, we propose SepHash, a write-optimized hash index for disaggregated memory. First, SepHash proposes a two-level separate segment structure that significantly reduces the bandwidth consumption of resize operations. Second, SepHash employs a low-latency concurrency control strategy to eliminate unnecessary mutual exclusion and check overhead during insert operations. Finally, SepHash designs an efficient cache and filter to accelerate read operations. The evaluation results show that, compared to state-of-the-art distributed hash indexes, SepHash achieves a 3.3X higher write performance while maintaining comparable read performance.

Synthetic MRI-Assisted and Self-Supervised Adaptive Segmentation of Organs-at-Risk (OARs) in MRI-Based Radiation Therapy

This study proposes a self-supervised solution for OAR segmentation, combining patch-based adaptation and unsupervised synthesis of T2-weighted MRI data to finetune the segmentation model. The aim is to improve adaptation to patient anatomy, overcome limited annotated MRI data, and enhance the generalizability of automatic segmentation models for gynecological cancers. The study used a patch-based cycle consistent generative adversarial network (cycle-GAN) for unsupervised MRI synthesis from CT scans of 20 patients, and a residual U-Net model for OARs segmentation. The segmentation model was trained and validated on synthetic MRI (sMRI) of 103 and 25 patient scans respectively, then finetuned on 78 MRI scans from radiation therapy fractions of 15 additional patients through three-fold cross validation. Self-supervised adaptation was applied, incorporating affine and elastic deformations, intensity shifting, and scaling. The model was trained on 96 × 96 × 96 sub-volumes and validated on entire pelvic sections of the same images. A combination of Dice and weighted cross entropy (CE) losses, with weights assigned for bladder (1), small bowel (1), rectum (2), sigmoid (2), left femoral head (0), and right femoral head (0), was used for OAR segmentation. The performance was evaluated against the model trained only on a limited number of acquired MRI data, as well as sMRI pretrained models with encoder weight freezing and either equal weighting or soft-tissue adjusted weighting. Our sMRI-assisted approach showed improved performance for challenging pelvic OARs compared to the method using only the acquired MRI data. The self-supervised fraction-adaptive segmentation results indicated better performance in target soft-tissues when using at least one treatment fraction for organ-specific adaptation. Our framework leverages pre-existing CT planning data for gynecological cancers to enhance the segmentation performance of OARs during MR-guided adaptive treatments. This approach offers substantial benefits for the radiation therapy workflow, including reduced variability in per-fraction segmentation and clinical burden. Further studies that involve human expert evaluations will be conducted to assess the practicality of this approach in radiation therapy.

GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning

Training data pre-processing pipelines are essential to deep learning (DL). As the performance of model training keeps increasing with both hardware advancements (e.g., faster GPUs) and various software optimizations, the data pre-processing on CPUs is becoming more resource-intensive and a severe bottleneck of the pipeline. This problem is even worse in the cloud, where training jobs exhibit diverse CPU-GPU demands that usually result in mismatches with fixed hardware configurations and resource fragmentation, degrading both training performance and cluster utilization. We introduce GoldMiner, an input data processing service for stateless operations used in pre-processing data for DL model training. GoldMiner decouples data pre-processing from model training into a new role called the data worker. Data workers facilitate scaling of data pre-processing to anywhere in a cluster, effectively pooling the resources across the cluster to satisfy the diverse requirements of training jobs. GoldMiner achieves this decoupling in a fully automatic and elastic manner. The key insight is that data pre-processing is inherently stateless, thus can be executed independently and elastically. This insight guides GoldMiner to automatically extract stateless computation out of a monolithic training program, efficiently disaggregate it across data workers, and elastically scale data workers to tune the resource allocations across jobs to optimize cluster efficiency. We have applied GoldMiner to industrial workloads, and our evaluation shows that GoldMiner can transform unmodified training programs to use data workers, accelerating individual training jobs by up to 12.1x. GoldMiner also improves average job completion time and aggregate GPU utilization by up to 2.5x and 2.1x in a 64-GPU cluster, respectively, by scheduling data workers with elasticity.

Cost-Effective Strong Consistency on Scalable Geo-Diverse Data Replicas

The Raft algorithm maintains strong consistency across data replicas in Cloud. This algorithm places nodes, i.e., leader and follower, to serve read/write requests spanning geo-diverse sites. As the workload increases, Raft shall provide proportional scale-out performance. However, traditional scale-out techniques are bottlenecked in Raft with an exponentially increased performance penalty when provisioned sites exhaust local resources. To provide scalability in Raft, this paper presents a cost-effective mechanism that enables elastic auto scaling in Raft, called BW-Raft. BW-Raft extends the original Raft with the following abstractions: (1) secretary nodes that take over expensive log synchronization operations from the leader, relaxing the performance constraint on locks. (2) observer nodes that handle reads only, improving throughput for typical data intensive services. These abstractions are stateless, allowing elastic scale-out on unreliable yet cheap spot instances. In theory, we prove that BW-Raft can preserve the strong consistency guarantee from Raft at scale-out, handling 50X more nodes, compared to the original Raft. We have prototyped the BW-Raft on key-value services and evaluated it with many state-of-the-arts on Amazon EC2 and Alibaba Cloud. Our results show that within the same budget, BW-Raft incurs 5-7X less resource footprint increment than Multi-Raft. Using spot instances, BW-Raft can reduces costs by 84.5%, compared to Multi-Raft. In the real world experiments, BW-Raft improves goodput of the 95th-percentile SLO by 9X, thus, serves an alternative for distributed service scaling out with strong consistency.

Large is different: Nonmonotonic behavior of elastic range scaling in polymeric turbulence at large Reynolds and Deborah numbers.

We use direct numerical simulations to study homogeneous and isotropic turbulent flows of dilute polymer solutions at high Reynolds and Deborah numbers. We find that for small wave numbers k, the kinetic energy spectrum shows Kolmogorov-like behavior that crosses over at a larger k to a novel, elastic scaling regime, E(k) ∼ k-ξ, with ξ ≈ 2.3. We study the contribution of the polymers to the flux of kinetic energy through scales and find that it can be decomposed into two parts: one increase in effective viscous dissipation and a purely elastic contribution that dominates over the nonlinear flux in the range of k over which the elastic scaling is observed. The multiscale balance between the two fluxes determines the crossover wave number that depends nonmonotically on the Deborah number. Consistently, structure functions also show two scaling ranges, with intermittency present in both of them in equal measure.

Knowledge Extraction and Discovery about Web System Based on the Benchmark Application of Online Stock Trading System.

Predicting workload characteristics could help web systems achieve elastic scaling and reliability by optimizing servers' configuration and ensuring Quality of Service, such as increasing or decreasing used resources. However, a successful analysis using a simulation model and recognition and prediction of the behavior of the client presents a challenging task. Furthermore, the network traffic characteristic is a subject of frequent changes in modern web systems and the huge content of system logs makes it a difficult area for data mining research. In this work, we investigate prepared trace contents that are obtained from the benchmark of the web system. The article proposes traffic classification on the web system that is used to find the behavior of client classes. We present a case study involving workload analysis of an online stock trading application that is run in the cloud, and that processes requests from the designed generator. The results show that the proposed analysis could help us better understand the requests scenario and select the values of system and application parameters. Our work is useful for practitioners and researchers of log analysis to enhance service reliability.

Modeling and optimizing micro-service based cloud elastic management system

Cloud elasticity is a comprehensive solution that takes into account all running applications and resources, including the cloud management system. Cloud resources and applications are constantly changing in terms of capacity and behavior, implying a dynamic change in the architecture and characteristics of the cloud management system. Following the micro-service architecture pattern, the new trend in application modeling is to decompose it into components and make them standalone cooperated modules. By customizing the application operation modules to match new tasks, this design provides the application with rapid adaptation agility to changes in requirements. This paper proposes a full-stack micro-service-based elastic cloud management system that elastically scales and manages cloud resources. The proposed model focuses on the elastic scaling performance of micro-service management modules by analyzing cloud management in three areas: interactions, end-to-end delay, and communication. In addition, optimizing the decoupling and orchestration scheduling of micro-service cloud management system components to achieve elastic management scaling. To the best of our knowledge, no work has investigated cloud management as a workload required for elastic resource provisioning. In comparison to a monolithic architecture, the proposed micro-service-based model achieves 45% to 65% enhancement in elastic scaling performance.

ScaleFlux: Efficient Stateful Scaling in NFV

Network function virtualization (NFV) enables elastic scaling to middlebox deployment and management. Therefore, efficient stateful scaling is an important task because operators often need to shift traffic and the associated flow states across VNF instances to deal with time-varying loads. Existing NFV scaling methods, however, typically focus on one aspect of the scaling pipeline and does not offer an end-to-end scaling framework. This article presents ScaleFlux, a complete stateful scaling system that efficiently reduces flow-level latency and achieves near-optimal resource usage. ScaleFlux (1) monitors traffic load for each VNF instance and adopts a queue-based mechanism to detect load burstiness timely, (2) deploys a flow bandwidth predictor to predict flow bandwidth time-series with the ABCNN-LSTM model, and (3) schedules the necessary flow and state migration using the simulated annealing algorithm to achieve both flow-level latency guarantee and resource usage minimization. Testbed evaluation with a five-machine cluster shows that ScaleFlux reduces flow completion time by at least 8.7× for all the workloads and achieves near-optimal CPU usage during scaling.