Datacenter Workloads Research Articles

Nash Bargaining Solution Allocation is Not Suitable for Datacenter Jobs

The Nash Bargaining Solution (NBS) has been broadly suggested as an effective solution for the problem of fair allocation of multiple resources, namely bandwidth allocation in datacenters. In spite of being thoroughly studied, and provably strategy-proof for most scenarios, NBS-based allocation methods lack research on the strategic behavior of tenants in the case of proportionality of resource demands, which is common in datacenter workloads. We found that misbehavior is beneficial: by lying about bandwidth demands tenants can improve their allocations. We show that a sequence of selfish improvements leads to trivial demand vectors for all tenants. It essentially removes sharing incentives which are very important for datacenter networks. In this paper, we analytically prove that tenants can misbehave in 2- and 3- tenants cases. We show that misbehavior is possible in one recently proposed NBS-based allocation system if proportionality of demands is taken into account. Monte Carlo simulations were done for 2–15 tenants to show a misbehavior possibility and its impact on aggregated bandwidth. We propose to use another game-theoretic approach, namely Dominant Resource Fairness (DRF) to allocate bandwidth in the case of proportional demands. We show that this method performs significantly better than NBS after misbehavior.

CARB: A C-State Power Management Arbiter for Latency-Critical Workloads

Latency-critical workloads in datacenters have tight response time requirements to meet service-level agreements (SLAs). Sleep states (c-states) enable servers to reduce their power consumption during idle times; however entering and exiting c-states is not instantaneous, leading to increased transaction latency. In this paper we propose a c-state arbitration technique, CARB, that minimizes response time, while simultaneously realizing the power savings that could be achieved from enabling c-states. CARB adapts to incoming request rates and processing times and activates the smallest number of cores for processing the current load. CARB reshapes the distribution of c-states and minimizes the latency cost of sleep by avoiding going into deep sleeps too often. We quantify the improvements from CARB with memcached running on an 8-core Haswell-based server.

Through the Looking Glass - The 2017 Edition: Trends in Solid-State Circuits from ISSCC

The International Solid-State Circuits Conference (ISSCC) is the flagship conference of the IEEE Solid-State Circuits Society. This year, the theme for ISSCC 2017 is Intelligent Chips for a Smart World. Advances in solid-state circuits and systems continue to propel the ongoing fusion between the physical and virtual worlds. With the resulting growth in sensor deployment, data traffic, and data-center workloads, future systems must employ intelligent chips at all levels of the system stack to improve the efficiency at which we acquire, network, store, and process information. Modern applications centered around the Internet of Everything (IoE) and real-time data analytics are driving circuit and system designers toward new ways of leveraging the immense device density and processing power of modern technology. ISSCC covers a spectrum of design approaches in various technical areas and advances, broadly categorized as analog systems, analog-to-digital data-conversion systems, communication systems, digital systems, and innovative topics such as micromachines and microelectromechanical systems (MEMS), imagers, sensors, and biomedical devices, as well as forward-looking solutions that may be several years away from becoming commercial. The ten ISSCC technical subcommittees annually update their analyses of industry trends for the benefit of the community at large. This article summarizes some of these views in selected technical areas.

Bilateral Electricity Trade Between Smart Grids and Green Datacenters: Pricing Models and Performance Evaluation

Datacenter demand response is a promising approach for mitigating operational instability faced by smart grids. It enables significant potentials in peak load shedding and facilitates the incorporation of distributed generation and intermittent energy sources. This paper considers two key aspects toward real-time electricity pricing for eliciting demand response: 1) two-way electricity flow between smart grids and large datacenters with hybrid green generation capabilities and 2) the geo-distributed nature of large cloud systems, and hence the potential competition among smart grids that serve different datacenters of the cloud. We propose a pricing scheme tailored for geo-distributed green datacenters, from a multi-leader (smart grids) single-follower (cloud) game point of view. At the cloud side, in quest for scalability, robustness, and performance, the energy cost minimization problem is solved in a distributed manner, based on the technique of alternating direction method of multipliers. At the smart grid side, a practical equilibrium of the multi-leader single-follower pricing game is desired. To this end, we employ the technique of equilibrium problem with equilibrium constraints and exact linearization, to accurately transform the multi-leader single-follower pricing game, which is non-convex into a mixed integer linear system that can be readily solved. The effectiveness of the proposed solutions is evaluated based on the real datacenter workload traces and the IEEE 14-bus test systems with real generation and demand data.

Joint Energy Management Strategy for Geo-Distributed Data Centers and Electric Vehicles in Smart Grid Environment

According to some forecast models, electric vehicles (EVs) are expected to have a high level of penetration in the coming decades. To support the daily operation of EVs, charging is necessary. Generally, there are three main charging scenes in the future, i.e., charging at workplace, charging at home, and charging at commercial station. For a data center operator, the synchronous EV charging of the employees at the workplace during the working hours would incur an additional large demand charge. To avoid/reduce such demand charge, we investigate a joint energy management problem for geo-distributed data centers and EVs of the employees in this paper. Specifically, we intend to minimize the cost of a data center operator by jointly scheduling data center workload and EV charging under the given power limits, where the cost consists of electricity bill, revenue loss associated with workloads, and battery depreciation cost. We first formulate a total cost minimization problem with the consideration of heterogeneous demands of EVs and the given power limits. Since the formulated problem is a large-scale convex optimization problem with temporally-coupled and spatially-coupled constraints, we propose a distributed algorithm to solve it. Based on the proposed algorithm, we design a joint energy management strategy for geo-distributed data centers and EVs. Simulation results based on real-world traces show that the proposed strategy could reduce the cost for the data center operator by up to 5.324%.

A self‐organized load balancing mechanism for cloud computing

SummaryThe growth in computer and networking technologies over the past decades established cloud computing as a new paradigm in information technology. The cloud computing promises to deliver cost‐effective services by running workloads in a large scale data center consisting of thousands of virtualized servers. The main challenge with a cloud platform is its unpredictable performance. A possible solution to this challenge could be load balancing mechanism that aims to distribute the workload across the servers of the data center effectively. In this paper, we present a distributed and scalable load balancing mechanism for cloud computing using game theory. The mechanism is self‐organized and depends only on the local information for the load balancing. We proved that our mechanism converges and its inefficiency is bounded. Simulation results show that the generated placement of workload on servers provides an efficient, scalable, and reliable load balancing scheme for the cloud data center. Copyright © 2016 John Wiley & Sons, Ltd.

An experimental evaluation of datacenter workloads on low-power embedded micro servers

This paper presents a comprehensive evaluation of an ultra-low power cluster, built upon the Intel Edison based micro servers. The improved performance and high energy efficiency of micro servers have driven both academia and industry to explore the possibility of replacing conventional brawny servers with a larger swarm of embedded micro servers. Existing attempts mostly focus on mobile-class micro servers, whose capacities are similar to mobile phones. We, on the other hand, target on sensor-class micro servers, which are originally intended for uses in wearable technologies, sensor networks, and Internet-of-Things. Although sensor-class micro servers have much less capacity, they are touted for minimal power consumption (< 1 Watt), which opens new possibilities of achieving higher energy efficiency in datacenter workloads. Our systematic evaluation of the Edison cluster and comparisons to conventional brawny clusters involve careful workload choosing and laborious parameter tuning, which ensures maximum server utilization and thus fair comparisons. Results show that the Edison cluster achieves up to 3.5x improvement on work-done-per-joule for web service applications and data-intensive MapReduce jobs. In terms of scalability, the Edison cluster scales linearly on the throughput of web service workloads, and also shows satisfactory scalability for MapReduce workloads despite coordination overhead.

Taurus

Many distributed workloads in today's data centers are written in managed languages such as Java or Ruby. Examples include big data frameworks such as Hadoop, data stores such as Cassandra or applications such as the SOLR search engine. These workloads typically run across many independent language runtime systems on different nodes. This setup represents a source of inefficiency, as these language runtime systems are unaware of each other. For example, they may perform Garbage Collection at times that are locally reasonable but not in a distributed setting. We address these problems by introducing the concept of a Holistic Runtime System that makes runtime-level decisions for the entire distributed application rather than locally. We then present Taurus, a Holistic Runtime System prototype. Taurus is a JVM drop-in replacement, requires almost no configuration and can run unmodified off-the-shelf Java applications. Taurus enforces user-defined coordination policies and provides a DSL for writing these policies. By applying Taurus to Garbage Collection, we demonstrate the potential of such a system and use it to explore coordination strategies for the runtime systems of real-world distributed applications, to improve application performance and address tail-latencies in latency-sensitive workloads.

Taurus

Many distributed workloads in today's data centers are written in managed languages such as Java or Ruby. Examples include big data frameworks such as Hadoop, data stores such as Cassandra or applications such as the SOLR search engine. These workloads typically run across many independent language runtime systems on different nodes. This setup represents a source of inefficiency, as these language runtime systems are unaware of each other. For example, they may perform Garbage Collection at times that are locally reasonable but not in a distributed setting. We address these problems by introducing the concept of a Holistic Runtime System that makes runtime-level decisions for the entire distributed application rather than locally. We then present Taurus, a Holistic Runtime System prototype. Taurus is a JVM drop-in replacement, requires almost no configuration and can run unmodified off-the-shelf Java applications. Taurus enforces user-defined coordination policies and provides a DSL for writing these policies. By applying Taurus to Garbage Collection, we demonstrate the potential of such a system and use it to explore coordination strategies for the runtime systems of real-world distributed applications, to improve application performance and address tail-latencies in latency-sensitive workloads.

Elastic Power-Aware Resource Provisioning of Heterogeneous Workloads in Self-Sustainable Datacenters

While major Cloud service operators have taken various initiatives to operate their datacenters with renewable energy partially or completely, it is challenging to effectively utilize the renewable energy since its generation depends on dynamic natural conditions. In this paper, we propose and develop an elastic power-aware resource provisioning approach (ePower) for heterogeneous workloads in self-sustainable datacenters that completely rely on renewable energy. We aim to maximize the system goodput and control the system power consumption with respect to green power supply. ePower takes challenges and advantages of dynamic power supply, heterogeneous workload characteristics and QoS requirements, and automatically optimizes elastic resource allocations to workloads. The core of ePower design is a novel power-aware simulated annealing algorithm with fuzzy performance modeling for the efficient search of an optimal resource allocation. We have implemented ePower in a university cloud testbed hosting Gridmix2 and RUBiS benchmark applications. We utilize real weather data traces to simulate the green power generation and supply in the experiments. Experimental results demonstrate ePower can achieve near-to-optimal system performance while being resilient to dynamic power availability. It outperforms a representative resource provisioning approach for heterogeneous workloads by at least 24% in improving system goodput and 35 percent in reducing QoS violations.

Real-Time Energy Management for Cloud Data Centers in Smart Microgrids

Cloud service providers are typically faced with three significant problems when running their cloud data centers, i.e., rising electricity bills, growing carbon footprints, and unexpected power outages. To mitigate these issues, running cloud data centers in smart microgrids (SMGs) is a good choice, since SMGs can enhance the energy efficiency, sustainability, and reliability of electrical services. Thus, in this paper, we investigate the problem of energy management for cloud data centers in SMGs. To be specific, we would minimize the time average expected energy cost (including electricity bill, battery depreciation cost, the total generation cost of conventional generators, and revenue loss due to the unfinished workloads) with the consideration of three practical factors, i.e., the ramping constraints of backup generators, the charging and discharging efficiency parameters of batteries, and two kinds of data center workloads. A stochastic programming is formulated by integrating the constraints associated with workload allocation, electricity buying/selling, battery management, backup generators, and power balancing. To solve the stochastic programming problem, an online algorithm is designed, and the algorithmic performance is analyzed. Simulation results show the advantages of the designed algorithm over other baselines.

Improving consolidation of virtual machine based on virtual switching overhead estimation

In virtualized data centers, live virtual machine (VM) migration can increase energy efficiency by consolidating VMs on fewer servers. This problem is usually considered a Bin Packing Problem with server capacity constraints, such as CPU, memory and network bandwidth. In order to minimize the communication traffic within the data center network, existing research works used correlation-based strategy to consolidate VMs onto servers, which means that VMs with inter-traffic are consolidated as closely as possible, e.g. within a server or a rack. However, this strategy increases the traffic load of virtual switches on servers, and it causes a certain number of CPU cycles of servers to move traffic through virtual switches. A lack of consideration for the virtual switching overhead may increase the risk that VMs are not allocated enough resources, and consequently reduce VMs׳ performance. In this work, we conduct experiments to estimate the virtual switching overhead on server CPU resource, and based on the experiment results, we propose a virtual-switching-aware VM consolidation algorithm to address this problem. Experiments on representative data center workloads show that the overhead can occupy 10–30% of server׳s CPU resources. Additionally, our algorithm shows a much lower server capacity violation probability as compared with the baseline algorithm.

PIASA: A power and interference aware resource management strategy for heterogeneous workloads in cloud data centers

Cloud data centers have been progressively adopted in different scenarios, as reflected in the execution of heterogeneous applications with diverse workloads and diverse quality of service (QoS) requirements. Virtual machine (VM) technology eases resource management in physical servers and helps cloud providers achieve goals such as optimization of energy consumption. However, the performance of an application running inside a VM is not guaranteed due to the interference among co-hosted workloads sharing the same physical resources. Moreover, the different types of co-hosted applications with diverse QoS requirements as well as the dynamic behavior of the cloud makes efficient provisioning of resources even more difficult and a challenging problem in cloud data centers. In this paper, we address the problem of resource allocation within a data center that runs different types of application workloads, particularly CPU- and network-intensive applications. To address these challenges, we propose an interference- and power-aware management mechanism that combines a performance deviation estimator and a scheduling algorithm to guide the resource allocation in virtualized environments. We conduct simulations by injecting synthetic workloads whose characteristics follow the last version of the Google Cloud tracelogs. The results indicate that our performance-enforcing strategy is able to fulfill contracted SLAs of real-world environments while reducing energy costs by as much as 21%.

Profiling-Based Workload Consolidation and Migration in Virtualized Data Centers

Improving energy efficiency of data centers has become increasingly important nowadays due to the significant amounts of power needed to operate these centers. An important method for achieving energy efficiency is server consolidation supported by virtualization. However, server consolidation may incur significant degradation to workload performance due to virtual machine (VM) co-location and migration. How to reduce such performance degradation becomes a critical issue to address. In this paper, we propose a profiling-based server consolidation framework which minimizes the number of physical machines (PMs) used in data centers while maintaining satisfactory performance of various workloads. Inside this framework, we first profile the performance losses of various workloads under two situations: running in co-location and experiencing migrations. We then design two modules: (1) consolidation planning module which, given a set of workloads, minimizes the number of PMs by an integer programming model, and (2) migration planning module which, given a source VM placement scenario and a target VM placement scenario, minimizes the number of VM migrations by a polynomial time algorithm. Also, based on the workload performance profiles, both modules can guarantee the performance losses of various workloads below configurable thresholds. Our experiments for workload profiling are conducted with real data center workloads and our experiments on our two modules validate the integer programming model and the polynomial time algorithm.

Optimal Resource Provisioning and the Impact of Energy-Aware Load Aggregation for Dynamic Temporal Workloads in Data Centers

An important goal of data center providers is to minimize their operational cost, which reflected through the wear-and-tear cost and the energy consumption cost. In this paper, we present optimization formulations to minimize the cost of ownership in terms of server energy consumption and server- wear-and-tear cost under three different data center server setups (homogeneous, heterogeneous, and hybrid hetero-homogeneous clusters) for dynamic temporal workloads. Our studies show that the homogeneous model takes significantly less computational time than the heterogeneous model (by an order of magnitude). To com- pute optimal configurations in near real time for large-scale data centers, we propose two modes for using our models: aggregation by maximum (preserves workload deadline) and aggregation by mean (relaxes workload deadline). In addition, we propose two aggregation methods for use in each of the two modes: static (peri- odic) aggregation and dynamic (aperiodic)aggregation. We found that in the aggregation by maximum mode, dynamic aggregation resulted in cost savings of up to approximately 18% over the static aggregation. In the aggregation by mean mode, dynamic aggrega- tion saved up to approximately a 50% workload rearrangement compared with the static aggregationby mean mode.

High reliable real-time bandwidth scheduling for virtual machines with hidden Markov predicting in telehealth platform

Reliable and high-performance resource scheduling for Virtual Machines (VMs) in cloud can guarantee the efficiency of remote rescue with telehealth system. When a local disaster, e.g. earthquake and tsunami, happens in a densely populated area, the surging health care demand leads to the increasing workload in Data Centers (DCs) by storing and delivering a mass of patients’ information and real-time physiology signals. However, the current self-adaptive scheduling methods cannot provide a high-accuracy recognizing of the two conditions: urgency or normal, which would procrastinate the system into a high-performance status, while the best rescue time is lost. In this paper, we propose a Primary Node-based architecture for typical telehealth service on cloud, which takes into account both storage and delivery efficiency. We also design a novel algorithm to predicting and allocating the future bandwidth of all VMs in the telehealth service context. This method is able to dynamically adjust each parameter of a Hidden Markov Model (HMM) through collecting the historical information of the bandwidth workload. After we predict the future bandwidth consumption of VMs, a high-performance scheduling method is used to adjust the bandwidth to each VM for health care service. The simulation results prove that this algorithm provides a high-accurate prediction, which guides the allocating module to make decision before the request burst comes. Nevertheless, our algorithm improves the reliability of telehealth services for storing and delivering patients’ information among DCs.

SLA-based virtual machine management for heterogeneous workloads in a cloud datacenter

Efficient provisioning of resources is a challenging problem in cloud computing environments due to its dynamic nature and the need for supporting heterogeneous applications. Even though VM (Virtual Machine) technology allows several workloads to run concurrently and to use a shared infrastructure, still it does not guarantee application performance. Thus, currently cloud datacenter providers either do not offer any performance guarantee or prefer static VM allocation over dynamic, which leads to inefficient utilization of resources. Moreover, the workload may have different QoS (Quality Of Service) requirements due to the execution of different types of applications such as HPC and web, which makes resource provisioning much harder. Earlier work either concentrate on single type of SLAs (Service Level Agreements) or resource usage patterns of applications, such as web applications, leading to inefficient utilization of datacenter resources. In this paper, we tackle the resource allocation problem within a datacenter that runs different types of application workloads, particularly non-interactive and transactional applications. We propose an admission control and scheduling mechanism which not only maximizes the resource utilization and profit, but also ensures that the QoS requirements of users are met as specified in SLAs. In our experimental study, we found that it is important to be aware of different types of SLAs along with applicable penalties and the mix of workloads for better resource provisioning and utilization of datacenters. The proposed mechanism provides substantial improvement over static server consolidation and reduces SLA violations.

A comparative study on resource allocation and energy efficient job scheduling strategies in large-scale parallel computing systems

In the large-scale parallel computing environment, resource allocation and energy efficient techniques are required to deliver the quality of services (QoS) and to reduce the operational cost of the system. Because the cost of the energy consumption in the environment is a dominant part of the owner's and user's budget. However, when considering energy efficiency, resource allocation strategies become more difficult, and QoS (i.e., queue time and response time) may violate. This paper therefore is a comparative study on job scheduling in large-scale parallel systems to: (a) minimize the queue time, response time, and energy consumption and (b) maximize the overall system utilization. We compare thirteen job scheduling policies to analyze their behavior. A set of job scheduling policies includes (a) priority-based, (b) first fit, (c) backfilling, and (d) window-based policies. All of the policies are extensively simulated and compared. For the simulation, a real data center workload comprised of 22385 jobs is used. Based on results of their performance, we incorporate energy efficiency in three policies i.e., (1) best result producer, (2) average result producer, and (3) worst result producer. We analyze the (a) queue time, (b) response time, (c) slowdown ratio, and (d) energy consumption to evaluate the policies. Moreover, we present a comprehensive workload characterization for optimizing system's performance and for scheduler design. Major workload characteristics including (a) Narrow, (b) Wide, (c) Short, and (d) Long jobs are characterized for detailed analysis of the schedulers' performance. This study highlights the strengths and weakness of various job scheduling polices and helps to choose an appropriate job scheduling policy in a given scenario.

Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Centers

Decreasing the soaring energy cost is imperative in large data centers. Meanwhile, limited computational resources need to be fairly allocated among different organizations. Latency is another major concern for resource management. Nevertheless, energy cost, resource allocation fairness, and latency are important but often contradicting metrics on scheduling data center workloads. Moreover, with the ever-increasing power density, data center operation must be judiciously optimized to prevent server overheating. In this paper, we explore the benefit of electricity price variations across time and locations. We study the problem of scheduling batch jobs to multiple geographically-distributed data centers. We propose a provably-efficient online scheduling algorithm - GreFar - which optimizes the energy cost and fairness among different organizations subject to queueing delay constraints, while satisfying the maximum server inlet temperature constraints. GreFar does not require any statistical information of workload arrivals or electricity prices. We prove that it can minimize the cost arbitrarily close to that of the optimal offline algorithm with future information. Moreover, we compare the performance of GreFar with ones of a similar algorithm, referred to as T-unaware, that is not able to consider the server inlet temperature in the scheduling process. We prove that GreFar is able to save up to 16 percent of energy-fairness cost with respect to T-unaware.

Datacenter Applications in Virtualized Networks: A Cross-Layer Performance Study

Datacenter-based Cloud computing has induced new disruptive trends in networking, key among which is network virtualization. Software-Defined Networking overlays aim to improve the efficiency of the next generation multitenant datacenters. While early overlay prototypes are already available, they focus mainly on core functionality, with little being known yet about their impact on the system level performance. Using query completion time as our primary performance metric, we evaluate the overlay network impact on two representative datacenter workloads, Partition/Aggregate and 3-Tier. We measure how much performance is traded for overlay's benefits in manageability, security and policing. Finally, we aim to assist the datacenter architects by providing a detailed evaluation of the key overlay choices, all made possible by our accurate cross-layer hybrid/mesoscale simulation platform.