Future Data Centers Research Articles

High-Efficiency High-Power-Density LLC Converter With an Integrated Planar Matrix Transformer for High-Output Current Applications

Isolated high-output current DC/DC converters are critical for future data center power architecture. LLC converters with matrix transformer are suitable for these applications due to its high efficiency and high power density. Different matrix transformer structures are investigated in this paper. To improve the current design practice, a high-frequency transformer loss model is developed and a detailed design methodology is proposed. Moreover, a novel matrix transformer structure is proposed to integrate four elemental transformers into one magnetic core with simple four-layer print circuit board windings implementation and further reduced core loss. By pushing switching frequency up to megahertz with GaN devices, the proposed matrix transformer can achieve high efficiency, high power density, and automatic manufacturing for magnetic components. A 1-MHz 380 V/12 V 800-W LLC converter with GaN devices is demonstrated. The prototype achieves a peak efficiency of 97.6% and a power density of 900 W/in3.

Future Energy Efficient Data Centers With Disaggregated Servers

The popularity of the Internet and the demand for 24/7 services uptime is driving system performance and reliability requirements to levels that today's data centers can no longer support. This paper examines the traditional monolithic conventional server (CS) design and compares it to a new design paradigm: the disaggregated server (DS) data center design. The DS design arranges data centers resources in physical pools, such as processing, memory, and IO module pools, rather than packing each subset of such resources into a single server box. In this paper, we study energy efficient resource provisioning and virtual machine (VM) allocation in DS-based data centers compared to CS-based data centers. First, we present our new design for the photonic DS-based data center architecture, supplemented with a complete description of the architectural components. Second, we develop a mixed integer linear programming (MILP) model to optimize VM allocation for the DS-based data center, including the data center communication fabric power consumption. Our results indicate that, in DS data centers, the optimum allocation of pooled resources and their communication power yields up to 42% average savings in total power consumption when compared with the CS approach. Due to the MILP high computational complexity, we developed an energy efficient resource provisioning heuristic for DS with communication fabric (EERP-DSCF), based on the MILP model insights, with comparable power efficiency to the MILP model. With EERP-DSCF, we can extend the number of served VMs, where the MILP model scalability for a large number of VMs is challenging. Furthermore, we assess the energy efficiency of the DS design under stringent conditions by increasing the CPU to memory traffic and by including high noncommunication power consumption to determine the conditions at which the DS and CS designs become comparable in power consumption. Finally, we present a complete analysis of the communication patterns in our new DS design and some recommendations for design and implementation challenges.

Competitive Evaluation of Planar Embedded Glass and Polymer Waveguides in Data Center Environments

Optical printed circuit board (OPCB) waveguide materials and fabrication methods have advanced considerably over the past 15 years, giving rise to two classes of embedded planar graded index waveguide based on polymer and glass. We consider the performance of these two emerging waveguide classes in view of the anticipated deployment in data center environments of optical transceivers based on directly modulated multimode short wavelength VCSELs against those based on longer wavelength single-mode photonic integrated circuits. We describe the fabrication of graded index polymer waveguides, using the Mosquito and photo-addressing methods, and graded index glass waveguides, using ion diffusion on thin glass foils. A comparative characterization was carried out on the waveguide classes to show a clear reciprocal dependence of the performance of different waveguide classes on wavelength. Furthermore, the different waveguide types were connected into an optically disaggregated data switch and storage system to evaluate and validate their suitability for deployment in future data center environments.

$\chi ^{(2)}$ Modulator With 40-GHz Modulation Utilizing BaTiO3 Photonic Crystal Waveguides

Future telecommunication and data center networks as well as quantum optical communication systems will require optical modulators with wide bandwidths, large extinction, low operating voltage, and small size. We report the first quantitative demonstration of slow light enhancement of the electro-optic (EO) coefficient in a χ (2) ferroelectric waveguide at microwave modulation frequencies. This is demonstrated in a compact (1 mm) photonic crystal (PC) device with a voltage-length product (Vπ·L) of 0.66 V-cm at 10 GHz and measured EO modulation out to 40 GHz. A local enhancement factor of 12 and effective EO coefficient of 900 pm/V is measured in the PC region at a wavelength of 1530 nm. By further optimizing the PC structure, devices with EO 3-dB bandwidths greater than 40 GHz and voltage-length product of 0.16 V-cm are predicted with 100-μm interaction length.

Resource Allocation in Electrical/Optical Hybrid Switching Data Center Networks

Hybrid switching combines the strengths of electronic packet switching and high-speed optical circuit switching and is thus widely believed to be a cost-effective solution for current and future data center networks (DCNs). In designing a hybrid switching DCN, it is essential to know how many resources should be devoted to optical circuit switching and electronic packet switching so the performance requirements in both planes can be satisfied, and, at the same time, objectives such as minimizing the overall system building cost or the power consumption can be realized. In this paper, we introduce BLOC, the Blocking LOss Curve, as a tool to characterize hybrid switching DCNs. With BLOC, the three interacting components in hybrid switching systems-resource allocation, traffic partitioning, and system performance-can be naturally integrated and studied together. We show how BLOC can be used to obtain feasible resource allocation/traffic partition combinations. We also show how system parameters, such as flow arrival characteristics, may affect resource allocation and system costs. Our study reveals that the total amount of resource required to satisfy 95th percentile traffic can be as low as 60% of that required for 99.9th percentile traffic.

DGLB: Distributed Stochastic Geographical Load Balancing over Cloud Networks

Contemporary cloud networks are being challenged by the rapid increase of user demands and growing concerns about global warming, due to their substantial energy consumption. This requires future data centers to be both energy efficient and sustainable, which calls for leveraging cutting-edge features and the flexibility provided by the modern smart grids. To fulfill those goals, this paper puts forward a systematic approach to designing energy-aware traffic-efficient geographical <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">load balancing schemes</i> for data-center networks that are not only optimal, but also computationally efficient and amenable to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">distributed</i> implementation. Under this comprehensive approach, workload and power balancing schemes are designed jointly across the network, both delay-tolerant and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interactive workloads</i> are accommodated, novel smart-grid features such as energy storage units are incorporated to cope with renewables, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">incentive pricing</i> mechanisms are adopted in the design. To further account for the spatio-temporal variation of demands, energy prices and renewables, the task is formulated as a two-timescale stochastic optimization. Leveraging dual stochastic approximation and the fast iterative shrinkage-thresholding algorithm (FISTA), the proposed optimization is decomposed across time slots (first-stage) and data centers (second-stage). While the resultant online algorithm is strictly feasible and provably optimal under a Markovian assumption for the underlying random processes, extensive numerical tests further demonstrate that it also works well in real-data scenarios, where the underlying randomness is highly correlated across time.

Analysis and Cell-Level Experimental Verification of a 25 kW All-SiC Isolated Front End 6.6 kV/400 V AC-DC Solid-State Transformer

Solid-state transformers (SSTs) could serve as interfaces between a medium-voltage (MV) AC grid and a low-voltage (LV) DC load or source, i. e., could be employed in applications with power supply character such as traction auxiliary supplies or rack-level power supplies in future datacenters. For handling the high input-side AC voltage and output side current, SSTs are typically realized as input-series output-parallel (ISOP) arrangements of multiple converter cells, whereby each cell comprises a medium-frequency isolation stage. This paper presents such a multi-cell 25 kW all-SiC MVAC-LVDC SST (6.6 kV AC to 400 V DC) based on the isolated front end (IFE) approach, which is an interesting alternative to the isolated back end (IBE) configuration mainly discussed in literature so far. The IFE concept is briefly explained, the main component stresses are derived, and a converter cell prototype is designed and tested. The 5 kW prototype cell features a power density of 1.5 kW/l (24.6 W/in3) and a measured peak efficiency of 97.5%. This is significantly higher than previously published data for 'FEbased SSTs, and in the same range as what has been reported recently for industrial IBE-based SSTs. Thus, this paper confirms that the 'FE approach can be a feasible and interesting alternative for realizing MVAC-LVDC SST systems with low complexity.

Reach Extension and Capacity Enhancement of VCSEL-Based Transmission Over Single-Lane MMF Links

This paper reviews and examines several techniques for expanding the carrying capacity of multimode fiber (MMF) using vertical cavity surface emitting lasers (VCSELs). The first approach utilizes short wavelength division multiplexing in combination with MMF optimized for operation between 850 and 950 nm. Both nonreturn to zero (NRZ) and four-level pulse amplitude modulation (PAM4) signaling are measured and demonstrate up to 170-Gb/s postforward error correction transmission over 300 m. For single wavelength transmission, the use of selective modal launch to increase the optical bandwidth of a standard OM3 MMF to more than 2.1 GHzckm for standard MMF is presented. A statistical model is used to predict the bandwidth enhancement of installed MMF and indicates that significant link extension can be achieved using selective modal launch techniques. These results demonstrate the continued effectiveness of VCSEL-based MMF links in current and future data center environments.

A 10 W On-Chip Switched Capacitor Voltage Regulator With Feedforward Regulation Capability for Granular Microprocessor Power Delivery

Granular power delivery with per-core regulation for microprocessor power delivery has the potential to significantly improve the energy efficiency of future data centers. On-chip switched capacitor converters can enable such granular power delivery with per-core regulation given a high efficiency, high power density, fast response time, and high output power converter design. This paper details the implementation of an on-chip switched capacitor voltage regulator in a $32\,\mathrm{n}\mathrm{m}$ SOI CMOS technology with deep trench capacitors. A novel feedforward control for reconfigurable switched capacitor converters is presented. The feedforward control reduces the output voltage droop following a transient load step. This leads to a reduced minimum microprocessor supply voltage, thereby reducing the overall power consumption of the microprocessor. The implemented on-chip switched capacitor voltage regulator provides a $0.7-1.1$ V output voltage from $1.8$ V input. It achieves a $85.1\%$ maximum efficiency at $3.2\,\mathrm{W}\mathrm{/}\mathrm{m}\mathrm{m}^2$ power density, a subnanosecond response time with improved minimum supply voltage capability, and a maximum output power of $10\,\mathrm{W}$ . For an output voltage of $850\,\mathrm{m}\mathrm{V}$ , the feedforward control reduces the required voltage overhead by $60\,\mathrm{m}\mathrm{V}$ for a transient load step from $10\%$ to $100\%$ of the nominal load. This can reduce the overall power consumption of the microprocessor by $7\%$ .

A Scalable, Partially Configurable Optical Switch for Data Center Networks

Optical circuit switching may be instrumental in meeting the cost, energy, and aggregate bandwidth requirements of future data center networks. However, conventional MEMS beam-steering cross-connects cannot provide submillisecond switching with the port count necessary for data centers. Here, we investigate a novel noncrossbar selector switch architecture and pupil-division switching layout to improve optical switching performance by relaxing the requirement of arbitrary switch configurability. This architecture and switch design enable MEMS beam-steering micromirrors to scale to microsecond response speeds while supporting high port count and low loss switching, and can realize a number of useful interconnection topologies. We present the design, fabrication, and experimental characterization of a proof-of-principle prototype using a single comb-driven MEMS mirror to achieve 150 μs switching of 61 ports between four preprogrammed interconnection mappings. We demonstrate the scalability of this switch architecture with a detailed optical design of a 2048-port selector switch with 20 μs switching time.

Low-power DAC-less PAM-4 transmitter using a cascaded microring modulator.

Future super-computer interconnect systems and data centers request ultrahigh data rate links at low cost and power consumption, for which transmitters with a high level of integration and spectral efficient formats are key components. We report 60 Gb/s pulse-amplitude modulation (PAM-4) of an optical signal using a dual-microring silicon photonics circuit, making a low-power, digital-to-analog converter (DAC)-less PAM modulator. The power consumption is evaluated below 100 fJ/bit, including thermal adjustments. To the best of our knowledge, these results feature the lowest reported power consumption for PAM signaling in a DAC-less scheme for data rate beyond 40 Gb/s.

Consequences of Future Data Center Deployment in Canada on Electricity Generation and Environmental Impacts: A 2015–2030 Prospective Study

SummaryThe environmental impacts of data centers that provide information and communication technologies (ICTs) services are strongly related to electricity generation. With the increasing use of ICT, many data centers are expected to be built, causing more absolute impacts on the environment. Given that electricity distribution networks are very complex and dynamic systems, an environmental evaluation of future data centers is uncertain. This study proposes a new approach to investigate the consequences of future data center deployment in Canada and optimize this deployment based on the Energy 2020 technoeconomic model in combination with life cycle assessment methodology. The method determines specific electricity sources that will power the future Canadian data centers and computes related environmental impacts based on several indicators. In case‐study scenarios, the largest deployment of data centers leads to the smallest impact per megawatt of data centers for all of the environmental indicators. It is found that an increase in power demand by data centers would lead to a reduction in electricity exports to the United States, driving the United States to generate more electricity to meet its energy demand. Given that electricity generation in the United States is more polluting than in Canada, the deployment of data centers in Canada is indirectly linked to an increase in overall environmental impacts. However, though an optimal solution should be found to mitigate global greenhouse gas emissions, it is not clear whether the environmental burden related to U.S. electricity generation should be attributed to the Canadian data centers.

Programming and Runtime Support to Blaze FPGA Accelerator Deployment at Datacenter Scale.

With the end of CPU core scaling due to dark silicon limitations, customized accelerators on FPGAs have gained increased attention in modern datacenters due to their lower power, high performance and energy efficiency. Evidenced by Microsoft's FPGA deployment in its Bing search engine and Intel's 16.7 billion acquisition of Altera, integrating FPGAs into datacenters is considered one of the most promising approaches to sustain future datacenter growth. However, it is quite challenging for existing big data computing systems-like Apache Spark and Hadoop-to access the performance and energy benefits of FPGA accelerators. In this paper we design and implement Blaze to provide programming and runtime support for enabling easy and efficient deployments of FPGA accelerators in datacenters. In particular, Blaze abstracts FPGA accelerators as a service (FaaS) and provides a set of clean programming APIs for big data processing applications to easily utilize those accelerators. Our Blaze runtime implements an FaaS framework to efficiently share FPGA accelerators among multiple heterogeneous threads on a single node, and extends Hadoop YARN with accelerator-centric scheduling to efficiently share them among multiple computing tasks in the cluster. Experimental results using four representative big data applications demonstrate that Blaze greatly reduces the programming efforts to access FPGA accelerators in systems like Apache Spark and YARN, and improves the system throughput by 1.7 × to 3× (and energy efficiency by 1.5× to 2.7×) compared to a conventional CPU-only cluster.

Modelling and dimensioning of a high-radix datacentre optical packet switch with recirculating optical buffers

Recently, high port-count optical packet switches, based on the Arrayed Waveguide Grating (AWG) device, have been proposed for providing the very high bandwidth and connectivity degree that will be required in future datacentre interconnection networks. A practical, cost-effective realisation of such a switch requires that over-subscription of the AWG cross-connect output ports be allowed, with the resulting packet contention being resolved using a packet buffering facility in the switch. In this paper, we propose an efficient AWG switch architecture of this type that uses a recirculating optical packet buffering scheme and we develop analytic modelling methods to estimate the switch packet blocking probability and packet buffering delay. The analytic model resolves internal traffic flows in the switch as Markov-modulated Poisson Processes (MMPP) and also allows modelling of offered load to the switch as a MMPP process. Taking this model of the switch, we first show its accuracy compared to discrete event simulations and then use it to develop a heuristic optimisation algorithm to dimension the switch hardware components, with the objective of minimising the operating cost by optimally dimensioning the main power consuming components, subject to constraints on the maximum contention probability and average and maximum propagation delays. The resulting optimised switch design demonstrates the proposed architecture to be a promising candidate for future high-port count datacentre switches.

Programmable Dynamically-Controlled Silicon Photonic Switch Fabric

Dynamically-controlled optical switches can enable novel functionalities in future high-performance computing and data center systems, including resource allocation and real-time routing. These capabilities can improve system robustness, performance, as well as overall power consumption. Silicon (Si) photonic switch fabrics are capable of nanosecond-scale switching, thus making them potentially attractive for dynamic reconfiguration at data packet rates, which are typically in the GHz range and are made via CMOS-compatible processes for smooth process integration. In this paper, we explore the dynamic control of data signals through an Si photonic switch matrix and demonstrate a programming interface for configuring the switch. Our associated control algorithm is shown to provide the capability for dynamically reallocating the optical power through multicasting for robust operation in case the optical loss changes along a given path. The scheme is shown to operate error-free with 40 Gb/s line rate data streams in a $4\times 4$ Si photonic switch fabric.

Efficient R-Tree Based Indexing Scheme for Server-Centric Cloud Storage System

Cloud storage system poses new challenges to the community to support efficient concurrent querying tasks for various data-intensive applications, where indices always hold important positions. In this paper, we explore a practical method to construct a two-layer indexing scheme for multi-dimensional data in diverse server-centric cloud storage system. We first propose RT-HCN, an indexing scheme integrating R-tree based indexing structure and HCN-based routing protocol. RT-HCN organizes storage and compute nodes into an HCN overlay, one of the newly proposed sever-centric data center topologies. Based on the properties of HCN, we design a specific index mapping technique to maintain layered global indices and corresponding query processing algorithms to support efficient query tasks. Then, we expand the idea of RT-HCN onto another server-centric data center topology DCell, discovering a potential generalized and feasible way of deploying two-layer indexing schemes on other server-centric networks. Furthermore, we prove theoretically that RT-HCN is both space-efficient and query-efficient, by which each node actually maintains a tolerable number of global indices while high concurrent queries can be processed within accepted overhead. We finally conduct targeted experiments on Amazon's EC2 platforms, comparing our design with RT-CAN, a similar indexing scheme for traditional P2P network. The results validate the query efficiency, especially the speedup of point query of RT-HCN, depicting its potential applicability in future data centers.

A Series-Stacked Power Delivery Architecture With Isolated Differential Power Conversion for Data Centers

In this paper, an alternative method to achieve more efficient dc power distribution and voltage regulation for future data centers is presented. This paper describes a series-stacked power delivery architecture, where servers are connected electrically in series, thereby, providing inherent step down of voltage. Server voltage regulation is performed by differential power converters, which only process the mismatch power between servers. The bulk power flows with no power processing, yielding greatly increased system efficiency compared to conventional architectures. We demonstrate the series-connected architecture with an experimental proof of concept and compare the proposed architecture with a conventional dc power delivery architecture employing a best-in-class power supply unit for servers. The proposed power delivery architecture is validated with a series-stacked server rack consisting of four 12 V servers, powered from a 48 V dc bus, performing two different real-world operations: web traffic management and computation. Through experimental measurements, we demonstrate up to a 40x reduction in power conversion losses compared to state-of-the-art hardware, and an overall best-case system conversion efficiency of 99.89%.

Cooling-Aware Energy and Workload Management in Data Centers via Stochastic Optimization

While the quest of end users for fast and convenient Internet services grows steadily, energy-hungry data centers correspondingly expand in both numbers and scale - a fact that raises global warming and climate change concerns. In addition, high penetration of renewables, development of energy-efficient cooling facilities, and flexibility of distributed storage units, all call for a system-wide energy and workload management policy for future sustainable data centers. As implementing offline management policies is practically infeasible due to complexity and the lack of future information, real-time management schemes are considered here under a systematic framework. Leveraging stochastic optimization tools, a unified management approach is proposed allowing data centers to adaptively respond to intermittent availability of renewables, variability of cooling efficiency, information technology (IT) workload shift, and energy price fluctuations under long-term quality-of-service (QoS) requirements. Meanwhile, it is rigorously established that when storage devices have sufficiently high capacity, or, the difference between electricity purchase and selling prices is small, the proposed algorithm yields a feasible and near-optimal management strategy without knowing the distributions of the independently and identically distributed (i.i.d.) workload, renewable, and electricity price processes. Numerical results further demonstrate that the proposed algorithm works well not only for i.i.d. processes, but also in real-data scenarios, where the underlying randomness is highly correlated over time.

Robust Workload and Energy Management for Sustainable Data Centers

A large number of geo-distributed data centers begin to surge in the era of data deluge and information explosion. To meet the growing demand in massive data processing, the infrastructure of future data centers must be energy-efficient and sustainable. Facing this challenge, a systematic framework is put forth in this paper to integrate renewable energy sources (RES), distributed storage units, cooling facilities, as well as dynamic pricing into the workload and energy management tasks of a data center network. To cope with RES uncertainty, the resource allocation task is formulated as a robust optimization problem minimizing the worst-case net cost. Compared with existing stochastic optimization methods, the proposed approach entails a deterministic uncertainty set where generated RES reside, thus can be readily obtained in practice. It is further shown that the problem can be cast as a convex program, and then solved in a distributed fashion using the dual decomposition method. By exploiting the spatio-temporal diversity of local temperature, workload demand, energy prices, and renewable availability, the proposed approach outperforms existing alternatives, as corroborated by extensive numerical tests performed using real data.

Capacity-enhanced receivers for low-latency Burst Optical Slot Switching rings

We propose a new receiver architecture for coherent detection in slotted optical packet switching rings with elastic (rate adaptive) optical transponders. Such rings are a candidate solution for future datacenter and metropolitan networks. The new receiver can detect more than a single packet per time slot and consequently has higher flexibility (translating into higher supported capacity, or, equivalently, lower end-to-end latency, or a combination or both), at the cost of a moderate increase in the transponder complexity and energy consumption (less than 10 %). We apply network planning and traffic engineering simulation tools (which we validate on small examples using theoretical models) to quantify the increase in network capacity and latency reduction that can be achieved thanks to the use of the new receivers. Finally, we identify the stability problem of the insertion process in the rings with the coherent receivers and propose a polynomial network planning algorithm, for the case of fast-tunable transmitters. We evaluate the cost of enforcing the stability, in terms of the additional transponders needed, for the mentioned case.