Data Center Operators Research Articles

Operations of data centers with onsite renewables considering greenhouse gas emissions

In a highly technology-dependent society, processing large amounts of data has become essential, resulting in a growing number of data centers. The global data center industry accounts for a significant proportion of the world’s energy consumption. A partial or total transition of the energy sources that power the operation of data centers to cleaner energy, particularly onsite renewable technologies, will help to achieve the greenhouse gas emission reduction goals established in international agreements. However, integrating such technologies is difficult due to the intermittency and availability of renewable sources. Nevertheless, temporal and geographical flexibility for processing part of the data centers’ workload provides an opportunity to efficiently integrate these renewable energy sources. In this paper, we develop an operational scheme for a network of interconnected data centers with a significant electricity supply from onsite technologies consisting of (i) an optimal job migration and scheduling policy for data centers to match their operation and the availability of renewable energy under an emissions cap-and-trade program; (ii) an energy management system to control the data centers’ onsite renewable and grid energy resources; and (iii) a robust optimization approach to account for the intermittency of the renewables and to identify the optimal migration and scheduling policies under the worst-case scenario. A case study is carried out to assess the economic and environmental impacts. The results indicate that the installation of onsite technologies and an optimal migration policy reduce the operational costs and the greenhouse gas emissions of the networks of data centers.

FG-PFC: A Fine-Grained PFC Mechanism for Lossless RDMA

Remote Direct Memory Access (RDMA) is widely deployed in data centers to improve the performance, efficiency, and reliability of data centers. Priority-based Flow Control (PFC) can ensure no packet loss as a necessary approach to achieve lossless RDMA. However, since PFC is a coarse-grained flow control mechanism based on port and priority, it can cause the head-of-line blocking problem and affect other flows, resulting in high latency. In this paper, we propose a fine-grained PFC mechanism (FG-PFC), which refines port and priority into flow granularity and further determines and pauses the flow most likely to cause congestion. The benefit is reducing the persistent high occupancy of the buffer and protecting the innocent flow. Simulations and experiments demonstrate that FG-PFC outperforms PFC in terms of flow completion time (FCT).

Progressive-fidelity computation of the genetic algorithm for energy-efficient virtual machine placement in cloud data centers

Energy efficiency is a critical issue in the management and operation of data centers, which form the backbone of cloud computing. Virtual machine placement has a significant impact on the energy efficiency of virtualized data centers. Among various methods to solve the virtual-machine placement problem, the genetic algorithm has been well accepted for its quality of solution. However, it is computationally demanding largely due to the complex form of fitness function, limiting its applications in data centers. To enhance the computational efficiency of the genetic algorithm while maintaining its quality of solution, a progressive-fidelity approach is developed in this paper for genetic-algorithm computation. It starts with a low-fidelity genetic algorithm with a simple fitness function. Then, for solution refinement, it switches to a medium-fidelity genetic algorithm with a more complicated fitness function. Finally, it progresses to the fine tuning of solution through a high-fidelity genetic algorithm with the energy consumption of data centers as fitness function. Heuristics are presented for the adaptive switching of genetic-algorithm computation from low fidelity to medium fidelity and finally to high fidelity. Experiments show that compared with the standard genetic algorithm of high fidelity, our progressive-fidelity approach of genetic algorithm computation is 50% faster for large-scale data centers while maintaining similar quality of solution in terms of the energy consumption of data centers.

Data center server workload and infrastructure control based on a joint RSM and CFD approach

The rapid expansion of data centers and cloud computing has exacerbated the issue of high energy consumption and localized overheating in servers. Consequently, the identification of effective cooling methods and resource allocation strategies for server rooms has become pivotal in enhancing overall airflow distribution and optimizing thermal performance within data centers. Time-efficient Computational Fluid Dynamics (CFD) tools offer an alternative to resource-intensive experimental measurements. However, it is difficult to monitor the information in real time and optimize the design of DC. In this study, an innovative application of Response Surface Methodology (RSM) is introduced to prediction thermal environment information and optimization performance of data center. In contrast to traditional single-factor optimization methods, this approach incorporates multiple factors, such as server workloads and air supply conditions, as optimization parameters. Furthermore, maximum server temperatures, temperature differences, and other metrics are defined as key performance indicators (KPIs). A predictive model was developed with the aim of offering comprehensive information about the thermal environment of the data center. Moreover, the study qualitatively analyzes the influence of each parameter on various indicators. Fitting equations are solved based on actual conditions to determine optimized configuration schemes in real time for the entire data center or specific scenarios. This approach effectively reduces cooling energy consumption and optimizes data center thermal management. The optimized configuration resulted in a significant reduction of approximately 30% in the maximum server temperature and approximately 20% in the temperature difference. Additionally, the optimization methodology established in this study facilitates the implementation of a real-time control mechanism for data center cooling systems, enabling energy demand and management optimization. The findings of this study offer valuable insights and generalized methods for enhancing the performance of air-cooled servers.

Entrainment of Mutually Synchronized Spatially Distributed 24 GHz Oscillators

Synchronization is one of the most challenging aspects of distributed systems in terms of their scalability. Minimal uncertainties can lead to problems or failures regarding data consistency in globally operating data centers or in distributed sensor arrays. Existing approaches to address these challenges are based on hierarchical synchronization concepts which are well understood and have reached technical maturity, but have the disadvantage of having a single point of failure. However, especially for critical infrastructure or backup more resilient solutions are required. Mutual synchronization where oscillators in a network are coupled bidirectionally without a reference have been considered. Due to the flat hierarchy such systems do not have a single point of failure. This work studies how hierarchical synchronization can be combined with architectures implementing mutual synchronization. A network of three mutually coupled 24 GHz oscillators is used to study how injecting a reference signal into one oscillator affects the dynamics. This can be quantified by analyzing in which range of frequencies the network of mutually coupled oscillators can follow the reference frequency. Measurements on a ring and chain network topology forced by an external reference oscillator shown here are in good agreement with the predictions of a nonlinear dynamical model.

데이터와 서비스 대범람의 시대, DR 데이터센터 원거리 배치를 위한 기술 및 정책 방안

Recently, in late 2022, Disaster Recovery Data Center is recognized as important due to the disruption issue of Kakao, the Korea' s #1 SNS service provider caused by a fire at their separated operating data center. In this response; the needs of establishing the Disaster Reovery Data Center, this paper is examining the brief current status of domestic and foreign data centers and D.R. data centers and analyzing the related problems. Moreover, according to three requirements (environment, communication and power infrastructure) and two criteria (site level, socio-economic/manpower/government) to find the r ight location for the data center, Jeju's strengths and weaknesses were examined, and considered with the technical and policy measures for Jeju Island to encourage for the attraction of private sector's data centers/DR data centers.

Memory Disaggregation: Advances and Open Challenges

Compute and memory are tightly coupled within each server in traditional datacenters. Large-scale datacenter operators have identified this coupling as a root cause behind fleetwide resource underutilization and increasing Total Cost of Ownership (TCO). With the advent of ultra-fast networks and cache-coherent interfaces, memory disaggregation has emerged as a potential solution, whereby applications can leverage available memory even outside server boundaries. This paper summarizes the growing research landscape of memory disaggregation from a software perspective and introduces the challenges toward making it practical under current and future hardware trends. We also reflect on our seven-year journey in the SymbioticLab to build a comprehensive disaggregated memory system over ultra-fast networks. We conclude with some open challenges toward building next-generation memory disaggregation systems leveraging emerging cache-coherent interconnects.

Virtualization Maturity in Creating System VM: An Updated Performance Evaluation

Virtualization technologies are indispensable in operating data centers and supporting cloud infrastructures, providing cost reduction (CapEx and OpEx), high availability, and disaster recovery. Hypervisor-assisted virtualization is one of the leading virtualization technologies, with the hypervisor being the software layer responsible for presenting the virtualized view of the hardware to system-level VMs. However, the virtualization overhead it introduces has implications into the computing infrastructure performance. This paper revisits key concepts about virtualization, technologies and techniques, types of VMs and hypervisors, and provides an up-to-date comparison between native and VM environments using workload metrics such as CPU and memory scores, disk speed, and network throughput to determine virtualization overhead. Our results show a clear overall trend toward meritorious performance and the maturity of the technologies used to create system-level VMs.

Reliability, availability, and life-cycle cost (LCC) analysis of combined cooling, heating and power (CCHP) integration to data centers considering electricity and cooling supplies

The utilization of digital technology is increasing rapidly and it is causing a growth in the number of data centers around the world. Data centers provide essential services, making them known as mission-critical. Like other mission-critical facilities, data centers often use reliability and availability analysis to evaluate the level of safety of their operations. Reliable electricity and cooling supplies are vital for the data center’s operation, and almost all data centers rely on the power grid to feed their electrical and cooling systems. However, the power grid is considered an unreliable source, and power grid fragility is a raising concern among data center operators. An alternative to the power grid is a sustainable combined cooling, heating, and power (CCHP) system, which can supply continuous electricity and cooling to a data center while providing local energy generation, energy cost saving, and emission benefits. To understand the impact of a CCHP integration on the reliability and availability of a grid-fed data center, an up-to-date electrical topology design and reliability block diagram models were created that take into account both electricity and cooling supplies. The results showed that the overall system reliability of a grid-fed data center increased from 49.23% to 74.62% with a CCHP integration. In terms of availability, a grid-fed data center with 2N redundant topology achieved the industry-accepted high availability level of 0.99999, also known as 5-nines. Moreover, the use of CCHP at the grid-fed data center increased the availability from 5-nines to 7-nines for the overall system. Thus, CCHP integration to a grid-fed data center provided higher reliability and availability for the overall system for even 2N redundant topology data centers. Moreover, based on the life-cycle cost (LCC) analysis, integrating CCHP systems into data centers has a promising outcome. However, when conducting an LCC analysis, we recommend considering the realistic cost of downtime along with the reliability-related exposure to properly assess the viability of alternative energy systems by data center investors and operators.

Towards Improving 5G Quality of Experience: Fuzzy as a Mathematical Model to Migrate Virtual Machine Server in The Defined Time Frame

The industry and government have recently acknowledged and used virtual machines (VM) to promote their businesses. During the process of VM, some problems might occur. The issues, such as a heavy load of memory, a large load of CPU, a massive load of a disk, a high load of network and time-defined migration, might interrupt the business processes. This paper identifies the migration process among hosts for VM to overcome the problem within the defined time frame of migration. The introduction of VMs migration in a timely manner is to detect a problem earlier. There are workload parameters, such as network, CPU, disk and memory as our parameters. To overcome the issue, we have to follow the Model named Fuzzy rule. The rule follows the basic of tree model for decision-making. The application of the fuzzy Model for the study is to determine VMs allocation from busy VMs to vacant VMs for balancing purposes. The result of the study showed that the use of the fuzzy Model to forecast VMs migration based on the defined rule had 2 positive impacts. The positive impacts are (1) Time frame live migration of VMs can reduce workload by 80 %. This aims to reduce failures in performing a live migration of VMs to increase data center performance. (2) In testing, the fuzzy Model can provide results with an accuracy of 90 %, so this model can perform a live migration of VMs precisely in determining the execution time. Next, the workload could be balanced among VMs. This research could be used further to improve 5G Quality of Experience (QoE) shortly.

The sustainability benefits of economization in data centers containing chilled water systems

This study provides the first known conservative estimates of reductions in carbon and water scarcity footprints when employing economization technologies in chilled water-based data centers (DCs) in the U.S. Specifically, a generic DC that employs a Computer Room Air Handling (CRAH)-based cooling system with a total IT load of 400 kW is computationally modeled at 925 locations in the U.S. with airside and/or water-side economization. The energy savings and environmental benefits are presented in terms of key DC performance metric reductions, including Carbon Usage Effectiveness (CUE) and Water Scarcity Usage Effectiveness (WSUE). The results suggest that implementing both water-side and airside economization schemes under conditions within the ASHRAE A1 Allowable envelope reduces the mean state-averaged PUE, WUE, CUE, and WSUE by 11%, 22%, 11%, and 15%, respectively. These results highlight the environmental benefits of economization technologies and provide guidance on economization strategies.

Resource optimization using predictive virtual machine consolidation approach in cloud environment

The Proliferation of on-demand usage-based IT services, as well as the diverse range of cloud users, have led to the establishment of energy-hungry hefty cloud data centers. Therefore, cloud service providers are striving to reduce energy consumption for cost-saving and environmental sustainability issues of data centers. In this direction, Virtual Machine (VM) consolidated is a widely used approach to optimize hardware resources at cost of performance degradation due to unnecessary migrations. Hence, the motivation of the proposed approach is to minimize energy consumption while maintaining the performance of cloud data centers. This leads to a reduction in the overall cost and an increase in the reliability of cloud service providers. To achieve this goal Predictive Virtual Machine Consolidation (PVMC) algorithm is proposed using the exponential smoothing moving average (ESMA) method. In the proposed algorithm, the ratio of deviation to utilization is calculated for VM selection and placement. migrating the high CPU using VMs or we can restrict steady resource-consuming VMs from migration. The outcomes of the proposed algorithm are validated on computer-based simulation under a dynamic workload and a variable number of VMs (1–290). The experimental results show an improvement in the mean threshing index (40%, 45%) and instruction energy ratio (15%, 17%) over the existing policies. Hence, the proposed algorithm could be used in real-world data centers for reducing energy consumption while maintaining low service level agreement violations.

მონაცემთა ცენტრის მომავალი, მისი დადებითი და უარყოფითი მხარეები

Why are more and more large and mid-sized organizations now considering data center outsourcing? Are the pros of IT outsourcing overblown? Or do organizations want to embrace the trend for the financial and operational advantages that data center management outsourcing offers? Unfortunately, a simple yes or no will not be enough to answer the above questions. According to a recent survey of Fortune 500 companies by Forbes, most IT companies now prefer IT outsourcing over in-house data center operations due to the advantages that data center outsourcing offers. This includes minimal operational costs, better utilization of infrastructure, access to servers and compute/storage on demand, etc. Before you decide to outsource your data center, carefully research your options, analyze your business prospects, and conduct a careful examination of your assets, budgets, and requirements. Carefully evaluating the pros and cons of data center outsourcing and how it might affect your business is something you should consider before signing an outsourcing contract. Data centers are critical to today's business scene. Although their demands have always been on an upward trajectory, the situation created by the COVID-19 pandemic has pushed it further. Like all other segments in and around IT, data center technologies are rapidly changing to achieve a highly optimized and efficient system. Therefore, it is very important for you as a business owner to stay up to date with each of these technological changes and trends. After all, they are going to create the future of data centers. Advances in technology will help the overall performance of the data center. Predictive analytics tools can help security and real-time monitoring teams spot potential problems early and fix them before they impact operational performance. Data center AI can highlight areas that could work more efficiently, such as cooling or power consumption. Thus, data center composability means creating resources that can work with many different systems. Instead of dedicated or narrow-use resources providing only one service, data centers should use resource pools that can contribute to the deployment of different types of workloads, such as storage, networking, or server equipment. Likewise, the implementation of composite resources helps to avoid locked capacity in the data center, thereby improving efficiency and throughput. Above all, digital transformation will enable data centers to deliver additional value to customers. Edge computing will help providers run low-latency workloads with end users, while software virtualization will create more budget-friendly options for multiple workloads. As private and public clouds combine to create more flexible and robust hybrid cloud deployments, companies will leverage technology to do more with less.

Coordinative Optimization Between Multiple Data Center Operators and a System Operator Based on Two-Level Distributed Scheduling Algorithm

Data Centers (DCs) have been playing a significant role in demand response (DR) programs in recent years due to their considerable DR capability. DCs are in the charge of the data center operator (DCO), who is responsible for making a reasonable allocation of computing tasks to provide DR resources. To relieve the transmission pressure of power systems, the system operator (SO) encourages DCOs to participate in the DR programs. To maximize the total welfare, a detailed DR scheduling model of DCOs and SO is proposed for the coordinative optimization. Considering the privacy issue of DCOs, a two-level distributed scheduling algorithm based on the alternating direction multiplier method (ADMM) is designed for privacy protection and distributed autonomy. Simulation results show that the proposed coordinative optimization algorithm can effectively realize the maximization of total social welfare with data privacy protection. For a power system with multiple DCOs, reasonable scheduling of DCO’s DR resources can reduce the peak-valley difference of system loads reliably and economically.

Predicting the state of health of VRLA batteries in UPS using data-driven method

Uninterruptible power battery (UPS) is an important part to ensure the stable operation of data center. Its security is related to the reliability and stability of power system. Among them, the state of health (SOH) prediction is a key issue of the valve regulated lead–acid (VRLA) battery operation and maintenance in data center. In this work, the battery SOH is predicted by the correlation between the nadir voltage value of Coup De Fouet (CDF) phenomenon and SOH. Then, the CDF phenomenon is combined with popular data-driven methods, such as linear regression, regression tree, support-vector machine, gaussian process, neural network, to predict battery SOH through 215 features. Finally, the above method is verified with the real discharge dataset of UPS battery in data center. The experimental results show that the data-driven method combining big data has higher accuracy than the simple prediction of battery SOH based on the nadir voltage value of CDF phenomenon and its variants.

A Virtual Data Center Comparison Of Different Firewalls' Performance

Whether virtual or real, every data centre depends on its network, and the firewall is a crucial component of that network for secure communication. Different types of firewalls, such as software firewalls, physical firewalls, virtual appliance firewalls, and kernel-integrated firewalls, can secure data centre connections. When choosing a firewall, there are many things to take into account, especially in a virtualized data centre where each firewall behaves differently depending on the situation. Reduced costs, effective administration extensibility, greater resource usage, scalability, and energy resilience are just a few benefits that virtualized data centres are supposed to produce. In this study, the use of firewalls is examined in relation to virtualized data centres. The effectiveness of several kinds of firewalls, including virtual firewalls, physical firewalls, and software .The performance of various types of firewalls, such as software firewalls, physical firewalls, virtual appliance firewalls, and kernel-integrated firewalls, is being analysed. Virtual data centre firewall implementation and performance comparisons explain how to design and which firewall type provides the best performance. In all conditions, it was shown that kernel-integrated firewalls worked properly. Virtual machine IP addresses and networks can vary, and the kernel-based firewall can dynamically update its rules to keep pace with such changes.

A New Approach to Capacity Scaling Augmented with Unreliable Machine Learning Predictions

Modern data centers suffer from immense power consumption. As a result, data center operators have heavily invested in capacity-scaling solutions, which dynamically deactivate servers if the demand is low and activate them again when the workload increases. We analyze a continuous-time model for capacity scaling, where the goal is to minimize the weighted sum of flow time, switching cost, and power consumption in an online fashion. We propose a novel algorithm, called adaptive balanced capacity scaling (ABCS), that has access to black-box machine learning predictions. ABCS aims to adapt to the predictions and is also robust against unpredictable surges in the workload. In particular, we prove that ABCS is [Formula: see text] competitive if the predictions are accurate, and yet, it has a uniformly bounded competitive ratio even if the predictions are completely inaccurate. Finally, we investigate the performance of this algorithm on a real-world data set and carry out extensive numerical experiments, which positively support the theoretical results. Funding: This work was partially supported by the Division of Computing and Communication Foundations [Grant 2113027]. The authors also acknowledge financial support for this project from the Algorithm and Randomness Center–Transdisciplinary Research Institute for Advancing Data Science Fellowship at Georgia Tech.

Efficient online migration mechanism for memory write-intensive virtual machines

Online migration of virtual machines (VMs) is indispensable for system maintenance as it helps to achieve several resource management objectives such as load balancing, proactive fault tolerance, green operation, and resource management of data centers. The migration efficiency and reliability are two major challenges in the online migration of memory write-intensive VMs. For example, pre-copy migration transfers a large amount of data and takes a long time to migrate. This study proposes an efficient and reliable adaptive hybrid migration mechanism for memory write-intensive VMs. The mechanism optimizes the data transfer mode of the common migration method and improves the performance of conventional hybrid migration. First, the virtual machine (VM) memory data to be migrated are divided into dynamic and static data based on the bitmap marking method, and the migration efficiency is improved through parallel transmission based on different networks. Second, to accelerate the migration reliability, an iterative convergence factor is proposed to evaluate the current system load state and adaptively calculate the switching time of the migration mode for adaptive hybrid migration based on the convergence factor. Through adaptive hybrid migration can achieve migration completed successfully, shorten the post-copy migration duration, and minimize the impact on the performance of VMs. Finally, this paper implements the system prototype based on a kernel-based virtual machine (KVM), and experiments are performed using multiple memory write-intensive load VMs. The results show that the proposed migration algorithm can significantly improve migration performance and complete migration quickly to solve the pre-copy migration failure problem with a memory write-intensive load. Compared with the traditional hybrid migration with only one round of pre-copy, the proposed migration algorithm reduces the total migration time and transmits data by 23.2% and 26.7%, respectively.

A Stochastic Approach to Determine the Optimal Number of Servers for Reliable and Energy Efficient Operation of Data Centers

The increasing demand of the data center's computational capacity in recent years has introduced new data center operational challenges among others to maintain the service level agreements (SLA) and quality of services (QoS), while at the same time limiting energy consumption. In this paper, a stochastic operational risk assessment approach is presented that estimates the required number of spare servers in a data center considering the risk of servers' failure in operation since servers define the computational capability of a data center. A reliability index called “risk of computational resource commitment (RCRC)” is introduced that quantifies the probability of having insufficient spare servers due to failures during the operational lead time, and the complement of the RCRC shows the ability of the resources to maintain SLA of a data center. The failure rates of the servers are obtained using a Monte Carlo Simulation with the failure data, published by Google in 2019. The analysis shows that the RCRC reduces with the increasing number of spare servers, while it also stresses the energy efficiency of the data center. The RCRC index could be used in data center operation to avoid overprovisioning of the servers and to limit the number of spare servers in the data center, while creating a suitable balance between QoS and energy consumption of the data centers.

Exploring the Impacts of Power Grid Signals on Data Center Operations Using a Receding-Horizon Scheduling Model

Data centers (DCs) can help decarbonize the power grid by helping absorb renewable power (e.g., wind and solar) due to their ability to shift power loads across space and time. However, to harness such load-shifting flexibility, it is necessary to understand how grid signals (carbon signals and market price/load allocations) affect DC operations. An obstacle that arises here is the lack of computationally-tractable DC operation models that can capture objectives, constraints, and information flows that arise at the interface of DCs and the power grid. To address this gap, we present a receding-horizon resource management model (a mixed-integer programming model) that captures the resource management layer between the DC scheduler and the grid while accounting for logical constraints, different types of objectives, and forecasts of incoming job profiles and of available computing capacity. We use our model to conduct extensive case studies based on public data from Microsoft Azure and MISO. Our studies show that DCs can provide significant temporal load-shifting flexibility that results in reduced carbon emissions and peak demand charges. Models and case studies are shared as easy-to-use Julia code.