Resource Utilization In Data Centers Research Articles

Optimizing Resource Management for Shared Microservices: A Scalable System Design

A common approach to improving resource utilization in data centers is to adaptively provision resources based on the actual workload. One fundamental challenge of doing this in microservice management frameworks, however, is that different components of a service can exhibit significant differences in their impact on end-to-end performance. To make resource management more challenging, a single microservice can be shared by multiple online services that have diverse workload patterns and SLA requirements. We present an efficient resource management system, namely Erms, for guaranteeing SLAs with high probability in shared microservice environments. Erms profiles microservice latency as a piece-wise linear function of the workload, resource usage, and interference. Based on this profiling, Erms builds resource scaling models to optimally determine latency targets for microservices with complex dependencies. Erms also designs new scheduling policies at shared microservices to further enhance resource efficiency. Experiments across microservice benchmarks as well as trace-driven simulations demonstrate that Erms can reduce SLA violation probability by 5× and more importantly, lead to a reduction in resource usage by 1.6×, compared to state-of-the-art approaches.

Quantifying Resource Contention of Co-located Workloads with the System-level Entropy

The workload co-location, such as deploying offline analysis workloads with online service workloads on the same node, has become common for modern data centers. Workload co-location deployment improves data center resource utilization significantly. Still, it also introduces resource contention, resulting in the online service’s quality of service fluctuation, which we call performance interference. As the online service is a tail-latency-sensitive workload, the tail-latency metric can reflect the performance interference degree of the co-location workloads at the application level. However, to guide system design and evaluation, quantitatively evaluating the resource contention of the co-located workloads at the system level is also essential. This article proposes a novel metric called System-Level Entropy (SLE). As a system-level metric, SLE can measure quantitatively resource contention of the co-location systems and perform the apples-to-apples comparison between systems. The experimental results show that SLE can accurately reflect the performance interference of workloads and then evaluate the system resource contention. We also demonstrate two case studies of the SLE. We quantify the affinity of different co-location combinations, including three online services and five offline workloads. Furthermore, we evaluate the effects of state-of-the-art isolated mechanisms (the container and the CPU–affinity binding) with these co-location combinations.

Autonomous DRL-based energy efficient VM consolidation for cloud data centers

The exponential data growth and demands for high computing resources lead to excessive resource use in cloud data centers, which cause an increase in energy consumption and high carbon emissions in the environment. So, the high energy consumption, inefficient resource usage, and quality of service assurance (QoS) are major challenges for cloud data centers. The dynamic consolidation of Virtual Machines (VMs) is proven to be an efficient way to tackle these issues while reducing energy consumption and improving resource utilization in data centers. It reduces the number of active hosts for energy efficiency by switching under-utilized and idle hosts into lower power mode. So, several heuristics and Artificial Intelligence (AI) based VM consolidation approaches have been published in papers. Most existing approaches rely on the aggressive consolidation of VMs for energy efficiency, thus causing performance degradation and high SLA violation. However, an automated solution is needed to reduce energy consumption and SLA violation by ensuring efficient resource usage in the cloud data center environment. Therefore, this article proposes an energy-efficient autonomous VM consolidation (AVMC) mechanism that has Deep Reinforcement Learning (DRL) based agent for performing VM consolidation decisions. The DRL agent learns the optimal distribution of VMs in the data center, considering energy efficiency and QoS assurance. The real-time workload traces from PlanetLab have been used to validate the proposed mechanism. Experimental results reveal the superiority of the proposed AVMC system over the existing models. AVMC reduced the energy consumption and SLA violation rate significantly.

Deep Reinforcement Learning for Provisioning Virtualized Network Function in Inter-Datacenter Elastic Optical Networks

In today’s datacenters (DCs), IT resources virtualization is leveraged to realize Network Function Virtualization (NFV) over general-purpose servers. Meanwhile, most of the service providers (SPs) are planning to use Virtual Network Functions (VNFs) to provide agile and flexible network services. In service provisioning, the VNF selection and mapping greatly affect IT resource utilization in DCs and spectrum resource utilization in optical networks. This paper proposes a Deep Reinforce Learning (DRL)-based algorithm for VNF provisioning. By selecting appropriate VNFs for the service requests, the algorithm intelligently guarantees efficient reusing of deployed VNFs while consuming fewer spectrum resources in inter-DC elastic optical networks (EONs). To facilitate the decision-making of the DRL agent, we first decompose the complex VNF-based service chaining (VNF-SC) into several VNF components (VNFCs), which can be solved one-by-one in turn. Then, a feature matrix-based encoding scheme is designed to represent the set of the VNFCs, the available DCs for the VNFCs, and the VNFC being operated, i.e., the input of neural networks. In addition, considering the complexity and difficulty of the VNF-SC provisioning problem, Double Deep Q Network (DDQN) is introduced in the proposed algorithm. Finally, compared with the benchmark heuristics, the extensive simulation results in different network topologies show that the proposed algorithm can reduce the IT and spectrum resource consumption by at least 9.6% and 1.6%, which proves the effectiveness of the proposed DRL-based VNF provisioning algorithm.

Multi-Objective Hybrid Flower Pollination Resource Consolidation Scheme for Large Cloud Data Centres

Cloud Computing has rapidly emerged as a successful paradigm for providing Information and Communication Technology (ICT) infrastructure. Resource allocation is used to execute user applications in the form of requests for consolidated resources in order to minimize energy consumption and violation of the Service Level Agreement (SLA) for large-scale data centers resource utilization. The energy consumption is usually caused due to local entrapment and violation of SLA during resource assigning and execution. Several researchers have proposed solutions to reduce local entrapments and violations of SLA, to minimize the energy consumption of the entire data center. However, strategies employed in their solutions face entrapment in either local searches or at the global search level with a certain level of SLA violation. In this light, a Multi-Objective Hybrid Flower Pollination Resource Consolidation (MOH-FPRC) scheme for efficient and optimal resource consolidation of data center resources is put forward. The Local Neighborhood Search (LNS) algorithm has been employed for addressing entrapment at the local search level, while the prominent flower pollination algorithm is used to solve the problem of entrapment at the global search level. This, in turn, reduces the energy consumption of the data centers. In addition, clustering strategies have been introduced with a robust migration mechanism to minimize the violation of SLA while also satisfying minimum energy consumption. The simulation results using the MultiRecCloudSim simulator have shown that our proposed MOH-FPRC demonstrates an improved performance on the data center energy consumption, resource utilization, and SLA violation with a 20.5% decrease, 23.9% increase, and 13.5% reduction, respectively, as compared with the benchmarked algorithms. The proposed scheme has proven its efficiency in minimizing energy consumption while at the same time improving the data center resource allocation with minimum SLA violations.

Redy

Redy is a cloud service that provides high performance caches using RDMA-accessible remote memory. An application can customize the performance of each cache with a service level objective (SLO) for latency and throughput. By using remote memory, it can leverage stranded memory and spot VM instances to reduce the cost of its caches and improve data center resource utilization. Redy automatically customizes the resource configuration for the given SLO, handles the dynamics of remote memory regions, and recovers from failures. The experimental evaluation shows that Redy can deliver its promised performance and robustness under remote memory dynamics in the cloud. We augment a production key-value store, FASTER, with a Redy cache. When the working set exceeds local memory, using Redy is significantly faster than spilling to SSDs.

Evaluation of Task Scheduling Algorithms in Heterogeneous Computing Environments.

This work establishes a set of methodologies to evaluate the performance of any task scheduling policy in heterogeneous computing contexts. We formally state a scheduling model for hybrid edge–cloud computing ecosystems and conduct simulation-based experiments on large workloads. In addition to the conventional cloud datacenters, we consider edge datacenters comprising smartphone and Raspberry Pi edge devices, which are battery powered. We define realistic capacities of the computational resources. Once a schedule is found, the various task demands can or cannot be fulfilled by the resource capacities. We build a scheduling and evaluation framework and measure typical scheduling metrics such as mean waiting time, mean turnaround time, makespan, throughput on the Round-Robin, Shortest Job First, Min-Min and Max-Min scheduling schemes. Our analysis and results show that the state-of-the-art independent task scheduling algorithms suffer from performance degradation in terms of significant task failures and nonoptimal resource utilization of datacenters in heterogeneous edge–cloud mediums in comparison to cloud-only mediums. In particular, for large sets of tasks, due to low battery or limited memory, more than 25% of tasks fail to execute for each scheduling scheme.

Energy-Aware Scheduling Based on Marginal Cost and Task Classification in Heterogeneous Data Centers

The energy consumption problem has become a bottleneck hindering further development of data centers. However, the heterogeneity of servers, hybrid cooling modes, and extra energy caused by system state transitions increases the complexity of the energy optimization problem. To deal with such challenges, in this paper, an Energy Aware Task Scheduling strategy (EATS) utilizing marginal cost and task classification method is proposed that cooperatively improves the energy efficiency of servers and cooling systems. An energy consumption model for servers, cooling systems, and state transition is developed, and the energy optimization problem in data centers is formulated. The concept of marginal cost is introduced to guide the task scheduling process. The task classification method is incorporated with the idea of marginal cost to further improve resource utilization and reduce the total energy consumption of data centers. Experiments are conducted using real-world traces, and energy reduction results are compared. Results show that EATS achieves more energy-savings of servers, cooling systems, state transition in comparison to the other two techniques under a various number of servers, cooling modules and task arrival intensities. It is validated that EATS is effective at reducing total energy consumption and improving the resource utilization of data centers.

Adaptive Preference-Aware Co-Location for Improving Resource Utilization of Power Constrained Datacenters

Large-scale datacenters often host latency-sensitive services that have stringent Quality-of-Service requirement and experience diurnal load pattern. Co-locating best-effort applications that have no QoS requirement with the latency-sensitive services has been widely used to improve the resource utilization of datacenters with careful shared resource management. However, existing co-location techniques tend to result in the power overload problem on power constrained servers due to the ignorance of the power consumption. To this end, we propose Sturgeon, a runtime system proactively manages resources between co-located applications in a power constrained environment, to ensure the QoS of latency-sensitive services while maximizing the throughput of best-effort applications. Our investigation shows that, at a given load, there are multiple feasible resource configurations to meet both QoS requirement and power budget, while one of them yields the maximum throughput of best-effort applications. To find such a configuration, we establish models to accurately predict the performance and power consumption of the co-located applications. Sturgeon monitors the QoS of the services periodically, in order to eliminate the potential QoS violation caused by the unpredictable interference. Besides, when the datacenter hosts different types of applications to perform co-location, Sturgeon places applications with their preferable candidates to improve the overall throughput. The experimental results show that at server level Sturgeon improves the throughput of the best-effort application by 25.43 percent compared to the state-of-the-art technique, while guaranteeing the 95%-ile latency within the QoS target; at cluster level, Sturgeon improves the overall throughput of best-effort applications by 13.74 percent compared to the baseline.

Traffic-profile and machine learning based regional data center design and operation for 5G network

Data center in the fifth generation (5G) network will serve as a facilitator to move the wireless communication industry from a proprietary hardware based approach to a more software oriented environment. Techniques such as Software defined networking (SDN) and network function virtualization (NFV) would be able to deploy network functionalities such as service and packet gateways as software. These virtual functionalities however would require computational power from data centers. Therefore, these data centers need to be properly placed and carefully designed based on the volume of traffic they are meant to serve. In this work, we first divide the city of Milan, Italy into different zones using K-means clustering algorithm. We then analyse the traffic profiles of these zones in the city using a network operator's Open Big Data set. We identify the optimal placement of data centers as a facility location problem and propose the use of Weiszfeld's algorithm to solve it. Furthermore, based on our analysis of traffic profiles in different zones, we heuristically determine the ideal dimension of the data center in each zone. Additionally, to aid operation and facilitate dynamic utilization of data center resources, we use the state of the art recurrent neural network models to predict the future traffic demands according to past demand profiles of each area.

Adaptive Prediction Models for Data Center Resources Utilization Estimation

Accurate estimation of data center resource utilization is a challenging task due to multi-tenant co-hosted applications having dynamic and time-varying workloads. Accurate estimation of future resources utilization helps in better job scheduling, workload placement, capacity planning, proactive auto-scaling, and load balancing. The inaccurate estimation leads to either under or over-provisioning of data center resources. Most existing estimation methods are based on a single model that often does not appropriately estimate different workload scenarios. To address these problems, we propose a novel method to adaptively and automatically identify the most appropriate model to accurately estimate data center resources utilization. The proposed approach trains a classifier based on statistical features of historical resources usage to decide the appropriate prediction model to use for given resource utilization observations collected during a specific time interval. We evaluated our approach on real datasets and compared the results with multiple baseline methods. The experimental evaluation shows that the proposed approach outperforms the state-of-the-art approaches and delivers 6% to 27% improved resource utilization estimation accuracy compared to baseline methods.

Adaptive sliding windows for improved estimation of data center resource utilization

Accurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous infrastructures, and multi-tenant co-hosted applications. Existing prediction methods use fixed size observation windows which cannot produce accurate results because of not being adaptively adjusted to capture local trends in the most recent data. Therefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick changing trends. In this paper we propose a deep learning-based adaptive window size selection method, dynamically limiting the sliding window size to capture the trend for the latest resource utilization, then build an estimation model for each trend period. We evaluate the proposed method against multiple baseline and state-of-the-art methods, using real data-center workload data sets. The experimental evaluation shows that the proposed solution outperforms those state-of-the-art approaches and yields 16 to 54% improved prediction accuracy compared to the baseline methods.

Optimistic scheduling with service guarantees

Data centers form the core of current information and commercial enterprise. At current scale, any improvement in data center resource utilization leads to substantial savings. We focus on the problem of scheduling jobs in distributed execution environments to improve resource utilization. Cluster schedulers like YARN and Mesos base their scheduling decisions on resource requirements provided by end users. It is hard for end-users to predict the exact amount of resources required for a task/job, especially since resource utilization can vary significantly over time and across tasks. In practice, users make highly conservative estimates of peak utilization across all tasks of a job to ensure job completion, leading to resource fragmentation and severe under utilization in production clusters. We present UBIS, a utilization-aware approach to cluster scheduling, to address resource fragmentation, and to improve cluster utilization and job throughput. UBIS considers actual usage of running tasks and schedules opportunistic work on under-utilized nodes. UBIS monitors resource usage on these nodes and preempts opportunistic containers in the event this over-subscription becomes untenable. In doing so, UBIS effectively utilizes wasted resources, while minimizing adverse effects on regularly scheduled tasks. Our implementation of UBIS on YARN demonstrates improvements of up to 30% in makespan for a representative workload and 25% in individual job durations.

Energy-Aware VM Consolidation in Cloud Data Centers Using Utilization Prediction Model

Virtual Machine (VM) consolidation provides a promising approach to save energy and improve resource utilization in data centers. Many heuristic algorithms have been proposed to tackle the VM consolidation as a vector bin-packing problem. However, the existing algorithms have focused mostly on the number of active Physical Machines (PMs) minimization according to their current resource requirements and neglected the future resource demands. Therefore, they generate unnecessary VM migrations and increase the rate of Service Level Agreement (SLA) violations in data centers. To address this problem, we propose a VM consolidation approach that takes into account both the current and future utilization of resources. Our approach uses a regression-based model to approximate the future CPU and memory utilization of VMs and PMs. We investigate the effectiveness of virtual and physical resource utilization prediction in VM consolidation performance using Google cluster and PlanetLab real workload traces. The experimental results show, our approach provides substantial improvement over other heuristic and meta-heuristic algorithms in reducing the energy consumption, the number of VM migrations and the number of SLA violations.

SLA-Aware and Energy-Efficient VM Consolidation in Cloud Data Centers Using Robust Linear Regression Prediction Model

Virtual machine (VM) consolidation provides a promising approach to save energy and to improve resource utilization in data centers. However, the aggressive consolidation of virtual machines may lead to service-level agreements (SLA) violation, which is essential for data centers and their users. Therefore, it is very meaningful to strike a tradeoff between power efficient and reduction of SLA violation level. In this paper, we propose a host overloading/underloading detection algorithm and a new VM placement algorithm based on our proposed robust simple linear regression prediction model for SLA-aware and energy-efficient consolidation of virtual machines in cloud data centers. Different from the native linear regression, our proposed methods amend the prediction and squint toward over-prediction by adding the error to the prediction; in this paper, we propose eight methods to calculate the error. We evaluate our proposed algorithms by extended the CloudSim simulator using PlanetLab workload and random workload. The experimental results show that our proposed model can reduce SLA violation rates by at most 99.16% and energy consumption by at most 25.43% for the real-world workload.

Confederated cloud—Design consideration for distributed utility computing system of systems

AbstractCloud computing advances are enabling cloud tenants to host and process their IT workloads in a safe, reliable, and cost‐effective manner. Federated clouds help cloud providers optimize many business parameters such as computing resource utilization. Federated clouds also introduce business risks to the cloud service providers in situations such as fewer tenants and various misconfigurations of energy, security, and load distributions arising from some possible sources. Yet, the rewards outweigh the risks when the federation implementation is successful. This work proposes a different optimization strategy that clarifies and simplifies the design trade‐offs for the federated clouds. We call it the confederated cloud. The strategy for it is to recruit customer groups by providing domain‐specific market services to cloud service providers that have a common market or business characteristics. The confederation will be a federation of domain groups, thus forming an interim cloud grouping that will retain domain‐specific privacy and governance structure and the confederation structure provide services on a more efficient or cost‐effective basis than can individual cloud service providers. This will help cloud providers to increase their datacenter resource utilization, decrease energy consumption, improve security, and increase revenue. It will also support cloud providers to set a suitable computing price for each computing resource type that matches the specific cloud group domain requirements. This solution will enable domain‐specific tenant groups to maintain controls of what is competitively vital and pool resources where it is advantageous to do so. The resulting architecture will be a decentralized system of systems that offers multicloud providers deployments similar to the centralized cloud federation. The confederated cloud system of systems described introduces a new system of systems engineering paradigm that uses domain‐specific governance for otherwise open communities that collaborate throughout the system life cycle. The confederated cloud system of systems has no explicit requirements for the cloud provider and cloud providers will not be aware of confederation services. Nonetheless, cloud providers might also not be aware of such system of systems. A successful confederated cloud will advantageous for many tenants. This enhanced market for cloud providers will help advance and drive the evolution of federated clouds.

Drawing the Cloud Map: Virtual Network Provisioning in Distributed Cloud Computing Data Centers

Efficient virtualization methodologies constitute the core of cloud computing data center implementation. Clients are attracted to the cloud model by its ability to scale the resources dynamically and the flexibility in payment options that it offers. However, performance hiccups may push them to go back to the buy-and-maintain model. Virtualization plays a key role in the synchronous management of the thousands of servers along with clients' data living on them. To achieve seamless virtualization, cloud providers require a system that performs the function of virtual network provisioning. This includes receiving the cloud client requests and allocating their computational and network resources in a way that guarantees the quality-of-service conditions for clients while maximizing the data center resource utilization and providers' revenue. We introduce a comprehensive system to solve the problem of virtual network mapping for a set of connection requests sent by cloud clients. Connections are collected in time intervals called windows. Consequently, node and link provisioning is performed. Different window size selection schemes are introduced and evaluated. Three schemes to prioritize connections are used, and their effect is assessed. Moreover, a technique dealing with connections spanning over more than a window is introduced. The proposed algorithm is compared with previous work well known in the literature. Simulation results show that the dynamic window size algorithm achieves cloud service providers' objectives in terms of generated revenue, served-connection ratio, resource utilization, and computational overhead. In addition, experimental results show that handling spanning connections independently improves the performance of the system.

Local Storage-Based Consolidation With Resource Demand Prediction and Live Migration in Clouds

Server consolidation is a useful solution aiming at cost-efficiency and high resource utilization of data centers and clusters. Nowadays, as the data-intensive and I/O intensive applications are widely used, more attention is paid to the local storage-based clouds which can offer much better I/O performance at relatively low price compared with the shared storage. However, it will obviously increase the migration cost (e.g., energy and time). Meanwhile, there is few suitable resource demand estimation method for local storage-based clouds at present, which plays an important role in system’s migration efficiency. And we find out that in this specific storage architecture, almost all the existing server consolidation algorithms do not have a suitable resource demand estimation method and a live migration scheme. To solve this problems, this paper designs and implements Combining Three (C3), a cloud architecture for local storage, C3 significant modules: prediction, consolidation, and migration. It was proved in statistical analysis that ARIMA may be the most suitable prediction model for the server workload, which motivates us to propose the resource estimation predictor. Also, we improve the existing consolidation method by adjusting the sorting index and fit degree during migration. Then, we propose a live migration scheme for local storage environment as the third module of C3. We conduct extensive experiments using real-world traces from Google to validate the effectiveness and superiority of our proposed algorithm.

Failure Analysis and Prediction for Big-Data Systems

Motivated by the high complexity of today’s datacenters, a large body of studies tries to understand workloads and resource utilization in datacenters. However, there is little work on exploring unsuccessful job and task executions. In this article, we study and predict three types of unsuccessful executions in traces of a Google datacenter, namely $\sf {fail}$ , $\sf {kill}$ , and $\sf {eviction}$ . We first quantitatively show their strongly negative impact on machine time and the resulting task slowdown. We analyze patterns of unsuccessful jobs and tasks, particularly focusing on their interdependencies, and we uncover their root causes by inspecting key workload and system attributes. Furthermore, we develop three on-line prediction models that can classify jobs and events into four classes upon arrival time, using independent or nested Neural Networks. We explore different combinations of feature sets and techniques to reduce the computational overhead. Our evaluation results show that the proposed models can accurately classify 94.4 percent of jobs and 76.8 percent of events into four classes.

An Adaptive Overload Detection Policy Based on the Estimator Sn in Cloud Environment

Efficient use of cloud resources and providing QoS to its clients is quite challenging for cloud service providers. On one hand, deployment of excessive active resources leads to increase in operational cost and on the other hand, shortage of resources may affect the QoS and SLA violations. In order to optimize the resource utilization of datacenter keeping SLA intact, the issues like over-loaded and under-loaded servers in a cloud datacenter are very important to deal with. Virtual machine migration technique is quite effective in handling such issues. The present work focuses on the adaptive threshold based overload detection policy which uses the robust estimator Sn for statistically analyzing the historical CPU usage of hosts, periodically and accordingly adjusts the upper CPU utilization threshold. The results obtained from proposed policy are compared with Median Absolute Deviation policy for overload detection and it has been found that energy performance efficiency of proposed policy is better than the median absolute deviation policy.