Real-world Data Center Research Articles

Swarm learning anomaly detection framework for cloud data center using multi-channel BiWGAN-GTN and CEEMDAN

Anomaly detection is an important task for maintaining the performance of cloud data center. Traditional anomaly detection primarily examines individual Virtual Machine (VM) behavior, neglecting the impact of interactions among multiple VMs on Key Performance Indicator (KPI) data,e.g., memory utilization. Furthermore, the non-stationarity, high complexity, and uncertain periodicity of KPI data in VM also bring difficulties to deep learning-based anomaly detection tasks. To settle these challenges, this paper proposes MCBiWGAN-GTN, a multi-channel semi-supervised time series anomaly detection algorithm based on the Bidirectional Wasserstein Generative Adversarial Network with Graph-Time Network (BiWGAN-GTN) and the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN). (a)The BiWGAN-GTN algorithm is proposed to extract spatiotemporal information from data. (b) The loss function of BiWGAN-GTN is redesigned to solve the abnormal data intrusion problem during the training process. (c) MCBiWGAN-GTN is designed to reduce data complexity through CEEMDAN for time series decomposition and utilizes BiWGAN-GTN to train different components. (d) To adapt the proposed algorithm for the entire cloud data center, a cloud data center anomaly detection framework based on Swarm Learning (SL) is designed. The evaluation results on a real-world cloud data center dataset show that MCBiWGAN-GTN outperforms the baseline, with an F1-score of 0.96, an accuracy of 0.935, a precision of 0.954, a recall of 0.967, and an FPR of 0.203. The experiments also verify the stability of MCBiWGAN-GTN, the impact of parameter configurations, and the effectiveness of the proposed SL framework.

E2SVM: Electricity-Efficient SLA-aware Virtual Machine Consolidation approach in cloud data centers.

Cloud data centers present a challenge to environmental sustainability because of their significant energy consumption. Additionally, performance degradation resulting from energy management solutions, such as virtual machine (VM) consolidation, impacts service level agreements (SLAs) between cloud service providers and users. Thus, to achieve a balance between efficient energy consumption and avoiding SLA violations, we propose a novel VM consolidation algorithm. Conventional algorithms result in unnecessary migrations when improperly selecting VMs. Therefore, our proposed E2SVM algorithm addresses this issue by selecting VMs with high load fluctuations and minimal resource usage from overloaded servers. These selected VMs are then placed on normally loaded servers, considering their stability index. Moreover, our approach prevents server underutilization by either applying all or no VM migrations. Simulation results demonstrate a 12.9% decrease in maximum energy consumption compared with the minimum migration time VM selection policy. In addition, a 47% reduction in SLA violations was observed when using the medium absolute deviation as the overload detection policy. Therefore, this approach holds promise for real-world data centers because it minimizes energy waste and maintains low SLA violations.

Long-term multivariate time series forecasting in data centers based on multi-factor separation evolutionary spatial–temporal graph neural networks

Data center infrastructures require constant monitoring to ensure stable and reliable operation and time-series forecasting plays an indispensable role in intelligent operations and maintenance in data centers. However, the potential for accurate time-series predictions is often limited due to the overlooked relationships between data records from independent sensors. Inferring relationships for a potential graph representation of a data center is challenging due to complex relationships between nodes and multiple factors that may cause connections between them. Moreover, graphs change dynamically in long-term predictions, but current methods do not account for future graph changes. To address these challenges, we propose a long-term time-series forecasting framework called Multi-factor Separation Evolutionary Spatial–Temporal Graph Neural Networks (MSE-STGNN). Our framework considers edge diversity, graph changes and spatial–temporal architecture in long-term prediction processes and proposes three modules. Specifically, we propose a Multi-factor Separation (MS) module to separate the factors influencing node connectivity, enabling the acquisition of a graph more closely aligned with actual circumstances; then we propose a Graph Prediction (GP) module to incorporate future graphs to correct errors in the graph on which multi-step predictions depend. Moreover, we propose an Attention-enhanced Spatial–temporal dilated causal convolution module (AS-Conv) to more effectively leverage information pertaining to spatial and historical events. Our experimental results on datasets comprising of temperature and IT power data collected from real-world data centers show that the proposed method outperforms other advanced prediction methods in terms of prediction accuracy, and the learned latent graphs are explainable.

Analysis of Energy and Network Cost Effectiveness of Scheduling Strategies in Datacentre

Abstract In parallel and distributed computing, cloud computing is progressively replacing the traditional computing paradigm. The cloud is made up of a set of virtualized resources in a data center that can be configured according to users’ needs. In other words, cloud computing faces the problem of a huge number of users requesting unlimited jobs for execution on a limited number of resources, which increases energy consumption and the network cost of the system. This study provides a complete analysis of classic scheduling techniques specifically for handling data-intensive workloads to see the effectiveness of the energy and network costs of the system. The workload is selected from a real-world data center. Moreover, this study offers the pros and cons of several classical heuristics-based job scheduling techniques that take into account the time and cost of transferring data from multiple sources. This study is useful for selecting appropriate scheduling techniques for appropriate environments.

Parameterized deep reinforcement learning with hybrid action space for energy efficient data center networks

To ensure the delivery of high-performance and reliable services, data center networks (DCNs) are often over-provisioned for peak workload and traffic bursts. However, in real-world data centers, network traffic seldom reaches peak capacity of the network, resulting in significant energy waste. Traditional energy conservation approaches either suffer from high computational complexity and low solution quality, or their strategies cannot be dynamically adjusted to accommodate changes in data center network traffic. Deep reinforcement learning (DRL) provides an effective way to deal with these issues. However, most of the existing DRL-based schemes only consider either a continuous action space or a discrete action space, which greatly restricts the optimality of decisions. To solve these problems, this paper proposes a novel DRL-based DCN energy optimization framework, named SmartDCN. Specifically, SmartDCN consists of a traffic prediction module (TPM) and an energy optimization module (EOM). TPM incorporates an improved LSTM model JANET with an attention mechanism providing a high prediction accuracy, while EOM integrates our newly proposed parameterized DRL algorithm, named PAS-DQN, combining with the discrete-continuous hybrid action space. PAS-DQN implements a two-level control mechanism for the network, using TPM to predict future traffic in the data center as input. It is devoted to dynamically aggregating current traffic and makes tradeoffs between energy efficiency, performance, and robustness to optimize the network’s power consumption by dynamically calculating the minimum required network subset and turning off the non-involved network devices to achieve power savings. Experimental results show that SmartDCN significantly outperforms the existing state-of-the-art schemes in terms of energy savings under various network conditions.

Optimal Rack-Coordinated Updates in Erasure-Coded Data Centers: Design and Analysis

Erasure coding has been extensively deployed in today's data centers to tackle prevalent failures, yet it is prone to substantial cross-rack traffic for parity updates. In this paper, we propose a new rack-coordinated update mechanism to suppress the cross-rack update traffic, which comprises two successive phases: a delta-collecting phase that collects data delta chunks, and another selective parity update phase that renews the parity chunks based on the update pattern and parity layout. We further design <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math>${\sf RackCU}$</tex-math></inline-formula> , an optimal rack-coordinated update solution that achieves the theoretical lower bound of the cross-rack update traffic. We also perform reliability analysis, demonstrating that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math>${\sf RackCU}$</tex-math></inline-formula> can attain a lower data loss probability via shortening the update procedure. We conduct extensive evaluations, in terms of large-scale simulation and real-world data center experiments. We show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math>${\sf RackCU}$</tex-math></inline-formula> can reduce 16.5-77.1% of the cross-rack update traffic and hence improve 24.9-772.0% of the update throughput.

A resource scheduling method for cloud data centers based on thermal management

With the rapid growth of cloud computing services, the high energy consumption of cloud data centers has become a critical concern of the cloud computing society. While virtual machine (VM) consolidation is often used to reduce energy consumption, excessive VM consolidation may lead to local hot spots and increase the risk of equipment failure. One possible solution to this problem is to utilize thermal-aware scheduling, but existing approaches have trouble realizing the balance between SLA and energy consumption. This paper proposes a novel method to manage cloud data center resources based on thermal management (TM-VMC), which optimizes total energy consumption and proactively prevents hot spots from a global perspective. Its VM consolidation process includes four phases where the VMs scheduler uses an improved ant colony algorithm (UACO) to find appropriate target hosts for VMs based on server temperature and utilization status obtained in real-time. Experimental results show that the TM-VMC approach can proactively avoid data center hot spots and significantly reduce energy consumption while maintaining low Service Level Agreement (SLA) violation rates compared to existing mainstream VM consolidation algorithms with workloads from real-world data centers.

Energy-Efficient Joint Task Assignment and Migration in Data Centers: A Deep Reinforcement Learning Approach

Energy-efficient task scheduling in data centers is a critical issue and has drawn wide attention. However, the task execution times are mixed and hard to estimate in a real-world data center. It has been conspicuously neglected by existing work that scheduling decisions made at tasks’ arrival times are likely to cause energy waste or idle resources over time. To fill in such gaps, in this paper, we jointly consider assignment and migration for mixed duration tasks and devise a novel energy-efficient task scheduling algorithm. Task assignment can improve resource utilization, and migration is required when long-running tasks run in low-load servers. Specifically: 1) We formulate mixed duration task scheduling as a large-scale Markov Decision Process (MDP) problem; 2) To solve such a large-scale MDP problem, we design an efficient Deep Reinforcement Learning (DRL) algorithm to make assignment and migration decisions. To make the DRL algorithm more practical in real scenarios, multiple optimizations are proposed to achieve online training; 3) Experiments with real-world data have shown that our algorithm outperforms the existing baselines 14% on average in terms of energy consumption while keeping the same level of Quality of Service (QoS).

Resource optimization using predictive virtual machine consolidation approach in cloud environment

The Proliferation of on-demand usage-based IT services, as well as the diverse range of cloud users, have led to the establishment of energy-hungry hefty cloud data centers. Therefore, cloud service providers are striving to reduce energy consumption for cost-saving and environmental sustainability issues of data centers. In this direction, Virtual Machine (VM) consolidated is a widely used approach to optimize hardware resources at cost of performance degradation due to unnecessary migrations. Hence, the motivation of the proposed approach is to minimize energy consumption while maintaining the performance of cloud data centers. This leads to a reduction in the overall cost and an increase in the reliability of cloud service providers. To achieve this goal Predictive Virtual Machine Consolidation (PVMC) algorithm is proposed using the exponential smoothing moving average (ESMA) method. In the proposed algorithm, the ratio of deviation to utilization is calculated for VM selection and placement. migrating the high CPU using VMs or we can restrict steady resource-consuming VMs from migration. The outcomes of the proposed algorithm are validated on computer-based simulation under a dynamic workload and a variable number of VMs (1–290). The experimental results show an improvement in the mean threshing index (40%, 45%) and instruction energy ratio (15%, 17%) over the existing policies. Hence, the proposed algorithm could be used in real-world data centers for reducing energy consumption while maintaining low service level agreement violations.

Design and Implementation of Smooth Renewable Power in Cloud Data Centers

The renewable power has been widely used in modern cloud data centers, which also produce large electricity bills and the negative impacts on environments. However, frequent fluctuation and intermittency of renewable power often cause the challenges in terms of the stability of both electricity grid and data centers, as well as decreasing the utilization of renewable power. Existing schemes fail to alleviate the renewable power fluctuation, which is caused by the essential properties of renewable power. In order to address this problem, we propose an efficient and easy-to-use smooth renewable power-aware scheme, called Smoother, which consists of Flexible Smoothing (FS) and Active Delay (AD). First, in order to smooth the fluctuation of renewable power, FS carries out the optimized charge/discharge operation via computing the minimum variance of the renewable power that is supplied to data centers per interval. Second, AD improves the utilization of renewable power via actively adjusting the execution time of deferrable workloads. Extensive experimental results via examining the traces of real-world data centers demonstrate that Smoother significantly reduces the negative impact of renewable power fluctuations on data centers and improves the utilization of renewable power by 250.88%. We have released the source codes for public use.

Reinforcement Learning for Datacenter Congestion Control

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. Evidently, the most popular recent deployments rely on rule-based heuristics that are tested on a predetermined set of benchmarks. Consequently, these heuristics do not generalize well to newly-seen scenarios. Contrarily, we devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We overcome challenges such as partial-observability, non-stationarity, and multi-objectiveness. We further propose a policy gradient algorithm that leverages the analytical structure of the reward function to approximate its derivative and improve stability. We show that these challenges prevent standard RL algorithms from operating within this domain. Our experiments, conducted on a realistic simulator that emulates communication networks' behavior, show that our method exhibits improved performance concurrently on the multiple considered metrics compared to the popular algorithms deployed today in real datacenters. Our algorithm is being productized to replace heuristics in some of the largest datacenters in the world.

Reinforcement Learning for Datacenter Congestion Control

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. Evidently, the most popular recent deployments rely on rule-based heuristics that are tested on a predetermined set of benchmarks. Consequently, these heuristics do not generalize well to newly-seen scenarios. Contrarily, we devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We overcome challenges such as partial-observability, nonstationarity, and multi-objectiveness. We further propose a policy gradient algorithm that leverages the analytical structure of the reward function to approximate its derivative and improve stability. We show that this scheme outperforms alternative popular RL approaches, and generalizes to scenarios that were not seen during training. Our experiments, conducted on a realistic simulator that emulates communication networks' behavior, exhibit improved performance concurrently on the multiple considered metrics compared to the popular algorithms deployed today in real datacenters. Our algorithm is being productized to replace heuristics in some of the largest datacenters in the world.

Performance analysis of a 1024-port Hipoλaos OPS in DCN, HPC, and 5G fronthauling Ethernet applications

The explosive traffic growth of emerging cloud, augmented/virtual reality, artificial intelligence, and 5G applications along with the inherent need for high-bandwidth transport of big data has fueled the expansion of the Ethernet switch market for data centers (DCs), high-performance computers (HPCs), and 5G fronthaul networks. As such, next-generation switches have to be capable of conforming to the requirements of a versatile traffic environment, expanding along DC, HPC, and 5G fronthauling infrastructures while meeting the performance needs of these different application sectors. Within this frame, we experimentally validate for the first time, to the best of our knowledge, the performance of the recently reported Hipoλaos optical packet switch (OPS) within Ethernet-based traffic exchange in DC and 5G fronthaul testbeds, highlighting the credentials of the 1024-port and 10.24 Tb/s capacity OPS to successfully support real-world DC and 5G applications. Error-free unicast and dual-output multicast Ethernet packet transmission at 10 Gb/s are successfully validated for different output ports of the switch, followed by successful server-to-server high-definition video transmission, both when the OPS was employed in the DC as well as in the 5G fronthauling testbed. Network performance and stability over time were confirmed through several measurements carried out through the iperf application suite, revealing submicrosecond end-to-end (Layer 7) latency performance. Finally, an OMNeT++ simulation analysis for an Ethernet-switched Hipoλaos network utilizing real-world application traces collected from the MareNostrum HPC system revealed up to 89% lower latency performance compared to the actual system.

A power and thermal-aware virtual machine management framework based on machine learning

Energy consumption in data centers grows rapidly in recent years. As a widely-applied energy-efficient method, workload consolidation also has its own limitations that may bring some negative effects, such as performance degradation, QoS violation, localized hotspots and so on, which is especially true when optimal objectives are inherently conflict. In this paper, we present a power and thermal-aware VM management framework called PTM-ML, which relies on machine learning technique to find optimal host configuration based on workload characteristics and cooling system’s working state. Based on such an optimal host configuration, it then makes VM migration and consolidation decisions by enforcing an efficient load-balancing policy, with aiming at achieving a better trade-off between energy efficiency and performance. The prototype of PTM-ML framework is deployed and evaluated in a real-world cloud data center. Extensive experiments are conducted by using different workload traces with distinctive characteristics, and the results are compared with four similar approaches in terms of total energy consumption, real-time power consumption, average latency and etc. Experimental results show that the proposed PTM-ML outperforms the existing approaches in terms of multiple metrics, and it also exhibits better robustness and adaptability in presence of dynamic workloads.

Online Dynamic B-Matching

This paper initiates the study of online algorithms for the maximum weight b-matching problem, a generalization of maximum weight matching where each node has at most b≥1 adjacent matching edges. The problem is motivated by emerging optical technologies which allow to enhance datacenter networks with reconfigurable matchings, providing direct connectivity between frequently communicating racks. These additional links may improve network performance, by leveraging spatial and temporal structure in the workload. We show that the underlying algorithmic problem features an intriguing connection to online paging (a.k.a. caching), but introduces a novel challenge. Our main contribution is an online algorithm which is O(b)- competitive; we also prove that this is asymptotically optimal. We complement our theoretical results with extensive trace-driven simulations, based on real-world datacenter workloads as well as synthetic traffic traces.

A global-energy-aware virtual machine placement strategy for cloud data centers

Virtual machine (VM) placement is a key technique for energy optimization in cloud data centers. Previous work generally focus on how to place the VMs efficiently in servers to optimize the physical resources used (e.g., memory, bandwidth, CPU, etc.), network resources used or cooling energy consumption. These work can optimize the energy consumption of cloud data centers according to one or two aspects (e.g. server, network or cooling), however, these methods may cause increased energy consumption in other aspects. To address this problem, we propose a global-energy-aware VMP (virtual machine placement) strategy to reduce, from multiple aspects, the total energy consumption of data centers. A two-step SAG algorithm is designed to lower the energy consumption of cloud data centers where multiple VMs are deployed. We conduct extensive experiments to evaluate the effectiveness of SAG. Two workloads from real-world data centers are utilized to quantitatively measure and compare the performance of our SAG with other typical algorithms. Experimental results indicate that, compared to other algorithms, our global-energy-aware VMP strategy can reduce the total energy consumption of the cloud data center by 8%–24.9%.

Fly-path: Traffic-based multi-hop routing approach for hybrid wireless data centers

High data transfer rates achieved by 802.11ad at 60 GHz ISM band enables use of wireless communication in data centers. In this paper, we investigate the possibility of offloading traffic from wired to wireless network in hybrid data centers. By understanding the capabilities of the wireless network, we can design the hybrid data center network accordingly, to achieve better construction and operating efficiency. First, we propose a system model in which each top-of-the-rack switch is equipped with two radios, so that three non-overlapping channels of 802.11ad that are available worldwide can be assigned in an interference-free manner to any configuration of wireless links. Then, we propose multi-hop routing algorithms that assign traffic to wireless infrastructure. These algorithms consist of two families. SP family of algorithms route traffic only over shortest-paths between source and destination pairs. LP algorithms relax this restriction and assign traffic to longer paths when necessary. In order to evaluate the performance of our routing algorithms, we also propose a random data center traffic generation method, based on an analysis of a real-world data center traffic pattern. We evaluate the performance of our allocation methods in terms of different metrics for various network sizes. Results show that our methods can offload significant amount of traffic from wired to wireless network, can achieve quite high throughput, and can utilize wireless links very well.

Machine Learning Data Center Workloads Using Generative Adversarial Networks

In this paper we study the applicability of generative adversarial networks (GANs) for the description and generation of workloads for data centers. GANs are advanced neural networks that can learn complex likelihood functions and can sample from them. The field of workload modeling is concerned with describing and generating realistic workloads for performance evaluation of computer systems, in this paper, specifically for data centers. The characterization of the workload of modern data centers is crucial in order to study the effect of changing workloads on the performance of such data centers. Previously, a number of statistical fitting techniques have been used to characterize data center workloads. This paper explores whether GANs are sufficiently capable to automatically learn such characterisations from multidimensional data sets. We describe the design and evaluation of a GAN, thereby using real-world data center traces. The learned model is evaluated by comparison to previously proposed fitting techniques. We find that the resulting GAN is very well able to reproduce a realistic data center workload. Furthermore, the approach does not require (a priori) knowledge or assumptions about the underlying models themselves, which can be seen as an advantage. It is shown that the learning approach does reach comparable quality to other fitting techniques, although still at much higher computational costs.

Development of a Precise and Cost-Effective Technique to Measure Deliquescent Relative Humidity of Particulate Contaminants and Determination of the Operating Relative Humidity of a Data Center Utilizing Airside Economization

Abstract A remarkable amount of energy in data centers is consumed in eliminating the heat generated by the information technology (IT) equipment to maintain and ensure safe operating conditions and optimum performance. The installation of airside economizers (ASEs), while very energy efficient, bears the risk of particulate contamination in data centers, hence, deteriorating the reliability of IT equipment. When relative humidity (RH) in data centers exceeds the deliquescent relative humidity (DRH) of salts or accumulated particulate matter, it absorbs moisture, becomes wet, and subsequently leads to electrical short-circuiting because of degraded surface insulation resistance (SIR) between the closely spaced features on printed circuit boards (PCBs). Another concern with this type of failure is the absence of evidence that hinders the process of evaluation and rectification. Therefore, it is imperative to develop a practical test method to determine the DRH value of the accumulated particulate matter found on PCBs. This research is a first attempt to develop an experimental technique to measure the DRH of dust particles by logging the leakage current versus RH% for the particulate matter dispensed on an interdigitated comb coupon. To validate this methodology, the DRH of pure salts like MgCl2, NH4NO3, and NaCl is determined, and their results are then compared with their published values. This methodology was therefore implemented to help lay a modus operandi of establishing the limiting value or an effective relative humidity envelope to be maintained at a real-world data center facility situated in Dallas industrial area for its continuous and reliable operation.

Identification and Characterization of Particulate Contaminants Found at a Data Center Using Airside Economization

Contamination due to the use of airside economizer has become a major issue that cost companies revenue. This issue will continue to rise as server components become smaller, densely packed, and as companies move into more polluted environments. Contaminants with small particles less than 10 μm are not noticeable; yet, these particles are most likely to get to areas where they can cause damage. Dust from different sources and suspended in air settles on surfaces of electrical components. The dust mainly contains two components: salts and metallic particles. The salts may be neutral or corrosive and the nature of the salt depends on the deliquescent humidity. For metallic particles, surveys are performed in various data centers in order to determine the limits in terms of weight per unit area and particle size distribution. It is necessary to first identify those contaminants that directly affect the information technology (IT) equipment in the data center. In this research, a real-world data center utilizing airside economization in an ANSI/ISA classified G2 environment was chosen for the study. Servers were removed and qualitative study of cumulative corrosion damage was carried out. The particulate contaminants were collected from different locations of a server and material characterization was performed using scanning electron microscopy (SEM), energy dispersive spectrometer (EDS), and Fourier transform infrared spectroscopy (FTIR). The analysis from these results helps to explain the impact of the contaminants on IT equipment reliability.