Future Server Research Articles

AV1: Implementation, Performance, and Application

With the finalization of the AVI specification, the Alliance for Open Media (AOM) has set high expectations for it. The prospect of a royalty-free alternative to high efficiency video coding (HEVC) has many supporters in the industry, particularly those seeking a standard for streaming applications that is optimized for over-the-top (OTT) delivery quite excited. Fundamental to AV1’s success will be the transfer of a toolset of efficient software implementations that can deliver the expected performance over existing compression standards within the constraints of future server provision. Efficient implementations of AV1 will require a careful balance between utilizing bespoke processor features to offload demanding processor-intensive functions and the desire to have a platform-agnostic compression standard that can be applied across a wide range of infrastructures. As with all complex compression schemes, realizing a software mapping of the toolset is crucial. This is true for AV1, which, being royalty-free, utilizes algorithms and methods that are different from those used in the Motion Picture Experts Group (MPEG) suite of standards. This article will look at the application challenges facing AV1 and compare its objective performance to that of the HEVC in terms of bitrate efficiency and encode time.

The effect of server energy proportionality on data center power oversubscription

Modern data centers improve resource utilization with power oversubscription. The power hierarchy in a data center is oversubscribed by installing more servers than allowed by the power budget based on server peak power consumption. Power oversubscription is possible due to the statistically low likelihood of simultaneous peak power operation of multiple servers. As future servers become more energy proportional, the opportunity for greater power oversubscription increases. The challenge is to quantify the level of oversubscription that can be attained. In this paper, we quantify the level of oversubscription possible for a given acceptable probability of power overload for servers characterized by energy proportionality metric and workload distribution. We develop a theoretical framework to characterize and predict the relationship between server energy proportionality and power oversubscription. We verify our framework through an extensive empirical study using publicly available SPECpower benchmark data for over 500 server models and publicly available Google cluster utilization data. Using our framework, a data center operator can predict the additional power oversubscription possible when replacing existing servers with a newer model of more energy proportional servers.

Fast Power and Energy Management for Future Many-Core Systems

Future servers will incorporate many active low-power modes for each core and for the main memory subsystem. Though these modes provide flexibility for power and/or energy management via Dynamic Voltage and Frequency Scaling (DVFS), prior work has shown that they must be managed in a coordinated manner. This requirement creates a combinatorial space of possible power mode configurations. As a result, it becomes increasingly challenging to quickly select the configuration that optimizes for both performance and power/energy efficiency. In this article, we propose a novel queuing model for working with the abundant active low-power modes in many-core systems. Based on the queuing model, we derive two fast algorithms that optimize for performance and efficiency using both CPU and memory DVFS. Our first algorithm, called FastCap, maximizes the performance of applications under a full-system power cap, while promoting fairness across applications. Our second algorithm, called FastEnergy, maximizes the full-system energy savings under predefined application performance loss bounds. Both FastCap and FastEnergy operate online and efficiently, using a small set of performance counters as input. To evaluate them, we simulate both algorithms for a many-core server running different types of workloads. Our results show that FastCap achieves better application performance and fairness than prior power capping techniques for the same power budget, whereas FastEnergy conserves more energy than prior energy management techniques for the same performance constraint. FastCap and FastEnergy together demonstrate the applicability of the queuing model for managing the abundant active low-power modes in many-core systems.

Mostly-optimistic concurrency control for highly contended dynamic workloads on a thousand cores

Future servers will be equipped with thousands of CPU cores and deep memory hierarchies. Traditional concurrency control (CC) schemes---both optimistic and pessimistic---slow down orders of magnitude in such environments for highly contended workloads. Optimistic CC (OCC) scales the best for workloads with few conflicts, but suffers from clobbered reads for high conflict workloads. Although pessimistic locking can protect reads, it floods cache-coherence backbones in deep memory hierarchies and can also cause numerous deadlock aborts. This paper proposes a new CC scheme, mostly-optimistic concurrency control (MOCC), to address these problems. MOCC achieves orders of magnitude higher performance for dynamic workloads on modern servers. The key objective of MOCC is to avoid clobbered reads for high conflict workloads, without any centralized mechanisms or heavyweight interthread communication. To satisfy such needs, we devise a native, cancellable reader-writer spinlock and a serializable protocol that can acquire, release and re-acquire locks in any order without expensive interthread communication. For low conflict workloads, MOCC maintains OCC's high performance without taking read locks. Our experiments with high conflict YCSB workloads on a 288-core server reveal that MOCC performs 8× and 23× faster than OCC and pessimistic locking, respectively. It achieves 17 million TPS for TPC-C and more than 110 million TPS for YCSB without conflicts, 170× faster than pessimistic methods.

Profiling a Warehouse-Scale Computer

Datacenters are quickly becoming the platform of choice for modern applications. In order to understand how datacenter software utilizes the hardware and to improve future server processor performance, the authors profiled more than 20,000 Google machines over a three-year period, while serving the requests of billions of users.

Designing Future Warehouse-Scale Computers for Sirius, an End-to-End Voice and Vision Personal Assistant

As user demand scales for intelligent personal assistants (IPAs) such as Apple’s Siri, Google’s Google Now, and Microsoft’s Cortana, we are approaching the computational limits of current datacenter (DC) architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this article, we present the design of Sirius, an open end-to-end IPA Web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of eight benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 8.5× and 15×, respectively. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of DCs by 2.3× and 1.3×, respectively.

Profiling a warehouse-scale computer

With the increasing prevalence of warehouse-scale (WSC) and cloud computing, understanding the interactions of server applications with the underlying microarchitecture becomes ever more important in order to extract maximum performance out of server hardware. To aid such understanding, this paper presents a detailed microarchitectural analysis of live datacenter jobs, measured on more than 20,000 Google machines over a three year period, and comprising thousands of different applications. We first find that WSC workloads are extremely diverse, breeding the need for architectures that can tolerate application variability without performance loss. However, some patterns emerge, offering opportunities for co-optimization of hardware and software. For example, we identify common building blocks in the lower levels of the software stack. This "datacenter tax" can comprise nearly 30% of cycles across jobs running in the fleet, which makes its constituents prime candidates for hardware specialization in future server systems-on-chips. We also uncover opportunities for classic microarchitectural optimizations for server processors, especially in the cache hierarchy. Typical workloads place significant stress on instruction caches and prefer memory latency over bandwidth. They also stall cores often, but compute heavily in bursts. These observations motivate several interesting directions for future warehouse-scale computers.

Sirius

As user demand scales for intelligent personal assistants (IPAs) such as Apple's Siri, Google's Google Now, and Microsoft's Cortana, we are approaching the computational limits of current datacenter architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this paper, we present the design of Sirius, an open end-to-end IPA web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of 7 benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 10x and 16x. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of datacenters by 2.6x and 1.4x, respectively.

Sirius

As user demand scales for intelligent personal assistants (IPAs) such as Apple's Siri, Google's Google Now, and Microsoft's Cortana, we are approaching the computational limits of current datacenter architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this paper, we present the design of Sirius, an open end-to-end IPA web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of 7 benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 10x and 16x. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of datacenters by 2.6x and 1.4x, respectively.

SBDP: Bandwidth prediction mechanism towards server demands in P2P-VoD system

Migrating popular videos into cloud is an efficient mechanism to decrease the costs of VoD services, but how to get the set of popular videos is a key problem. In this paper, we propose a prediction mechanism named SBDP to evaluate future server bandwidth demands on each video. Firstly, SBDP adopts time-series analysis techniques and history dataset to predict online population in advance. Then in order to obtain initial online population, the curve shape similar (CSS) and Gaussian process regression (GPR) methods are introduced. And finally a multi linear regression (MLR) method with the online population as a main factor is proved and applied in predicting seeds’ upload bandwidth. The proposed methods are simulated on large dataset collected from a popular VoD services in China and the simulation results demonstrate that the prediction data are close to real-world data.

Rhythm

Trends in increasing web traffic demand an increase in server throughput while preserving energy efficiency and total cost of ownership. Present work in optimizing data center efficiency primarily focuses on the data center as a whole, using off-the-shelf hardware for individual servers. Server capacity is typically increased by adding more machines, which is cheap, though inefficient in the long run in terms of energy and area. Our work builds on the observation that server workload execution patterns are not completely unique across multiple requests. We present a framework---called Rhythm---for high throughput servers that can exploit similarity across requests to improve server performance and power/energy efficiency by launching data parallel executions for request cohorts. An implementation of the SPECWeb Banking workload using Rhythm on NVIDIA GPUs provides a basis for evaluating both software and hardware for future cohort-based servers. Our evaluation of Rhythm on future server platforms shows that it achieves 4x the throughput (reqs/sec) of a core i7 at efficiencies (reqs/Joule) comparable to a dual core ARM Cortex A9. A Rhythm implementation that generates transposed responses achieves 8x the i7 throughput while processing 2.5x more requests/Joule compared to the A9.

Rhythm

Trends in increasing web traffic demand an increase in server throughput while preserving energy efficiency and total cost of ownership. Present work in optimizing data center efficiency primarily focuses on the data center as a whole, using off-the-shelf hardware for individual servers. Server capacity is typically increased by adding more machines, which is cheap, though inefficient in the long run in terms of energy and area. Our work builds on the observation that server workload execution patterns are not completely unique across multiple requests. We present a framework---called Rhythm---for high throughput servers that can exploit similarity across requests to improve server performance and power/energy efficiency by launching data parallel executions for request cohorts. An implementation of the SPECWeb Banking workload using Rhythm on NVIDIA GPUs provides a basis for evaluating both software and hardware for future cohort-based servers. Our evaluation of Rhythm on future server platforms shows that it achieves 4x the throughput (reqs/sec) of a core i7 at efficiencies (reqs/Joule) comparable to a dual core ARM Cortex A9. A Rhythm implementation that generates transposed responses achieves 8x the i7 throughput while processing 2.5x more requests/Joule compared to the A9.

Totally green

The environmental impact of servers and datacenters is an important future challenge. System architects have traditionally focused on operational energy as a proxy for designing green servers, but this ignores important environmental implications from server production (materials, manufacturing, etc.). In contrast, this paper argues for a lifecycle focus on the environmental impact of future server designs, to include both operation and production. We present a new methodology to quantify the total environmental impact of system design decisions. Our approach uses the thermodynamic metric of exergy consumption, adapted and validated for use by system architects. Using this methodology, we evaluate the lifecycle impact of several example system designs with environment-friendly optimizations. Our results show that environmental impact from production can be important (around 20% on current servers and growing) and system design choices can reduce this component (by 30--40%). Our results also highlight several, sometimes unexpected, cross-interactions between the environmental impact of production and operation that further motivate a total lifecycle emphasis for future green server designs.

Totally green

The environmental impact of servers and datacenters is an important future challenge. System architects have traditionally focused on operational energy as a proxy for designing green servers, but this ignores important environmental implications from server production (materials, manufacturing, etc.). In contrast, this paper argues for a lifecycle focus on the environmental impact of future server designs, to include both operation and production. We present a new methodology to quantify the total environmental impact of system design decisions. Our approach uses the thermodynamic metric of exergy consumption, adapted and validated for use by system architects. Using this methodology, we evaluate the lifecycle impact of several example system designs with environment-friendly optimizations. Our results show that environmental impact from production can be important (around 20% on current servers and growing) and system design choices can reduce this component (by 30--40%). Our results also highlight several, sometimes unexpected, cross-interactions between the environmental impact of production and operation that further motivate a total lifecycle emphasis for future green server designs.

Workload Performance Characterization and Test Strategy of High-Performance Fault-Tolerant Computers Based on BIBbench

It is critical to understand the workload characteristics and resource usage patterns of available applications to guide the design and development of architecture of the future large scale servers. In this paper, we analyze the workload performance characteristics of the actual Bank Intermediary Business (BIB) characteristics with BIBmodel and BIBbench design work, and propose BIB performance workload and use case definitions. The analysis and comparisons of workload and use case illustrate that the workload performance characteristics of BIB is totally different with TPC benchmarks. With the development of economy and technology, the requirements for BIB servers are important in modeling, benchmarks developing and workload performance characteristics studying are increased nowadays.

Trends in Server Energy Proportionality

Energy proportionality should be a key target in future server design. Although energy proportionality, as quantified by the proposed EP metric, has improved significantly in recent years, much more can be done to move systems closer to ideal energy proportionality. Toward this end, we are pursuing a workload characterization and large-scale system simulation methodology for advising data center system integrators to move toward cost and power-efficient infrastructures.

Gallium arsenide (GaAs) nanofibers by electrospinning technique as future energy server materials

Gallium arsenide (GaAs) does have superior electronic properties compared with silicon. For instant, it has a higher saturated electron velocity and higher electron mobility. Weak mechanical properties and high production cost are the main drawbacks of this interesting semiconductor. In this study, we are introducing production of GaAs nanofibers by electrospinning methodology as a very low cost and yielding distinct product technique. In general, nano-fibrous shape is strongly improving the physical properties due to the high surface area to volume ratio of this nanostructure. The mechanical and environmental properties of the GaAs compound have been modified since GaAs nanofibers have been produced as a core inside a poly(vinyl alcohol) (PVA) shell. GaAs/PVA nanofibers were prepared by electrospinning of gallium nitrate/PVA solution in presence of arsenic vapor. The whole process was carried out in a closed hood equipped with nitrogen environment. FT-IR, XPS, TGA and UV-Vis spectroscopy analyses were utilized to confirm formation of GaAs compound. Transmission electron microscope (TEM) analysis has revealed that the synthesized GaAs compound is crystalline and does have nano-fibrous shape as a core inside PVA nanofibers. To precisely recommend the prepared GaAs nanofiber mats to be utilized in different applications, we have measured the electric conductivity and the band gap energies of the prepared nanofiber mats. Overall, the obtained results affirmed that the proposed strategy successfully remedied the drawbacks of the reported GaAs structures and did not affect the main physical properties of this important semiconductor.

Disaggregated memory for expansion and sharing in blade servers

Analysis of technology and application trends reveals a growing imbalance in the peak compute-to-memory-capacity ratio for future servers. At the same time, the fraction contributed by memory systems to total datacenter costs and power consumption during typical usage is increasing. In response to these trends, this paper re-examines traditional compute-memory co-location on a single system and details the design of a new general-purpose architectural building block-a memory blade-that allows memory to be "disaggregated" across a system ensemble. This remote memory blade can be used for memory capacity expansion to improve performance and for sharing memory across servers to reduce provisioning and power costs. We use this memory blade building block to propose two new system architecture solutions-(1) page-swapped remote memory at the virtualization layer, and (2) block-access remote memory with support in the coherence hardware-that enable transparent memory expansion and sharing on commodity-based systems. Using simulations of a mix of enterprise benchmarks supplemented with traces from live datacenters, we demonstrate that memory disaggregation can provide substantial performance benefits (on average 10X) in memory constrained environments, while the sharing enabled by our solutions can improve performance-per-dollar by up to 57% when optimizing memory provisioning across multiple servers.

A new strategy to manage the InfiniBand arbitration tables

The InfiniBand Architecture (IBA) is an industry-standard architecture for server I/O and interprocessor communication. InfiniBand is extensively used for interconnection in high-performance clusters. It has been developed by the InfiniBand S M Trade Association (IBTA) to provide the levels of reliability, availability, performance, scalability, and quality of service (QoS) necessary for present and future server systems. The provision of QoS in data communication networks is currently the focus of much discussion and research in both industry and academia. In that sense, IBA enables QoS support with some mechanisms. In this paper, we examine these mechanisms and describe a way to use them. We propose a traffic segregation strategy based only on delay requirements. Moreover, we propose a very effective methodology to compute the virtual lane arbitration tables. Finally, we evaluate our proposal and performance results show that, with a correct traffic treatment at the output ports, every traffic class meets its QoS requirements.

3D DRAM in Server Application

Server system requirement drives the evolution of 3D DRAM technologies. The new generation provides form factor, electrical and power performance benefits. Meanwhile it brings in a more complicated design, new assembly technologies and new failure mechanisms. The optimum 3D DRAM technology is determined by complex tradeoff of benefits, cost, time to market and reliability. The characteristics of the optimum 3D DRAM technology for DDR2, DDR3 and future server memory system are analyzed in detail. TSV (Through Silicon Via) DRAM array stacking is expected to bring superior power performance which could be a significant market diver. The tradeoff of this future technology and process development trend is presented.