Server Processors Research Articles

Numerical reproducibility for the parallel reduction on multi- and many-core architectures

A parallel algorithm to compute correctly-rounded floating-point sumsHighly-optimized implementations for modern CPUs, GPUs and Xeon PhiAs fast as memory bandwidth allows for large sums with moderate dynamic rangeScales well with the problem size and resources used on a cluster of compute nodes On modern multi-core, many-core, and heterogeneous architectures, floating-point computations, especially reductions, may become non-deterministic and, therefore, non-reproducible mainly due to the non-associativity of floating-point operations. We introduce an approach to compute the correctly rounded sums of large floating-point vectors accurately and efficiently, achieving deterministic results by construction. Our multi-level algorithm consists of two main stages: first, a filtering stage that relies on fast vectorized floating-point expansion; second, an accumulation stage based on superaccumulators in a high-radix carry-save representation. We present implementations on recent Intel desktop and server processors, Intel Xeon Phi co-processors, and both AMD and NVIDIA GPUs. We show that numerical reproducibility and bit-perfect accuracy can be achieved at no additional cost for large sums that have dynamic ranges of up to 90 orders of magnitude by leveraging arithmetic units that are left underused by standard reduction algorithms.

Heterogeneity and thermal aware adaptive heuristics for energy efficient consolidation of virtual machines in infrastructure clouds

Holistic datacenter energy minimization operation should consider interactions between computing and cooling source specific usage patterns. Decisions like workload type, server configuration, load, utilization etc., contributes to power consumption and influences datacenter's thermal profile and impacts the energy required to control temperature within operational thresholds. In this paper, we present an adaptive virtual machine placement and consolidation approach to improve energy efficiency of a cloud datacenter; accounting for server heterogeneity, server processor low-power SLEEP state, state transition latency and integrated thermal controls to maintain datacenter within operational temperature. Our proposed heuristic approach reduces energy consumption with acceptable level of performance.

Optimal partitioning of a multicore server processor

Optimal partitioning of a multicore server processor in a cloud computing environment, i.e., optimal system (virtual server) configuration for some given types of applications is considered in this paper. Such optimization is important for dynamic resource provision and on-demand server customization in a cloud computing environment for certain specific types of applications, such that the overall system performance is optimized without exceeding certain energy consumption budget. A multicore server processor is treated as a group of queueing systems with multiple servers, i.e., M/M/m queueing systems. The system performance measures are the average task response time and the average power consumption. Two core speed and power consumption models are considered, namely, the idle-speed model and the constant-speed model. Three problems are formulated and solved, namely, optimal multicore server processor partitioning, optimal multicore server processor partitioning with power constraint, and optimal power allocation. All these problems are well-defined optimization problems. It is shown that although these problems are sophisticated, they can be solved by numerical algorithms. Numerical data are demonstrated for each problem.

A Simple Model to Quantify the Impact of Memory Latency and Bandwidth on Performance

In recent years, DRAM technology improvements have scaled at a much slower pace than processors. While server processor core counts grow from 33% to 50% on a yearly cadence, DDR4 memory channel bandwidth has grown at a slower rate, and memory latency has remained relatively flat for some time. Meanwhile, new computing paradigms have emerged, which involve analyzing massive volumes of data in real time and place pressure on the memory subsystem. The combination of these trends makes it important for computer architects to understand the sensitivity of the workload performance to memory bandwidth and latency. In this paper, we outline and validate a methodology for quick and quantitative performance estimation using a real-world workload.

Profiling a warehouse-scale computer

With the increasing prevalence of warehouse-scale (WSC) and cloud computing, understanding the interactions of server applications with the underlying microarchitecture becomes ever more important in order to extract maximum performance out of server hardware. To aid such understanding, this paper presents a detailed microarchitectural analysis of live datacenter jobs, measured on more than 20,000 Google machines over a three year period, and comprising thousands of different applications. We first find that WSC workloads are extremely diverse, breeding the need for architectures that can tolerate application variability without performance loss. However, some patterns emerge, offering opportunities for co-optimization of hardware and software. For example, we identify common building blocks in the lower levels of the software stack. This "datacenter tax" can comprise nearly 30% of cycles across jobs running in the fleet, which makes its constituents prime candidates for hardware specialization in future server systems-on-chips. We also uncover opportunities for classic microarchitectural optimizations for server processors, especially in the cache hierarchy. Typical workloads place significant stress on instruction caches and prefer memory latency over bandwidth. They also stall cores often, but compute heavily in bursts. These observations motivate several interesting directions for future warehouse-scale computers.

The HEROIC Framework: Encrypted Computation Without Shared Keys

Outsourcing computation to the cloud has recently become a very attractive option for enterprises and consumers, due to mostly reduced cost and extensive scalability. At the same time, however, concerns about the privacy of the data entrusted to cloud providers keeps rising. To address these concerns and thwart potential attackers, cloud providers today resort to numerous security controls as well as data encryption. Since the actual computation is still unencrypted inside cloud microprocessor chips, it is only a matter of time until new attacks and side channels are devised to leak sensitive information. To address the challenge of securing general-purpose computation inside microprocessor chips, we propose a novel computer architecture, and present a complete framework for general-purpose encrypted computation without shared keys, enabling secure data processing. This new architecture, called homomophically encrypted one instruction computation, contrary to the previous work in the area does not require a secret key installed inside the microprocessor chip. Instead, it leverages the powerful properties of homomorphic encryption combined with the simplicity of one instruction set computing. The proposed framework introduces: 1) a RTL implementation for reconfigurable hardware and 2) a ready-to-deploy virtual machine, which can be readily ported to existing server processor architectures.

Machine Learning Adaptation in Post Silicon Server Validation

As the complexity of modern server processors increases so the validation challenges. Current design validation methods cover less, resulting in bug escape and more regress postsilicon validation. The biggest problem is manual debugging of several failures by large number of test cases. By using machine learning in server validation, validation efforts and resource requirement will reduce. Validation of future generation server will be done through the learning set generated from the previous generation device, which is a set of test cases being passed. General Terms Algorithm, Machine learning, Validation.

A Mobile Application for Real-Time Multimodal Routing Under a Set of Users’ Preferences

In this article an application for mobile navigation is introduced with the intention to serve users who either do not have access to a private vehicle or prefer combining a vehicle with a public transport mode provided that it yields a reduction to their total travel time. This application guides users to their destination by integrating the use of private and public transport and differs from in-vehicle route guidance systems that suggest routes only for private vehicle users, and from other conventional mobile route guidance systems that propose either a route for public transport users or a route for private vehicle users. In addition, the proposed application suggests intermodal routes that comply with each individual's preferences in an attempt to fulfill the user's needs in complex, urban networks. The data requirements for the mobile application are covered by a real-time database that is handled by a central server (processor) and contains information about traffic congestion, train and bus schedules, and more. Finally, two algorithms are presented in order to serve the computational part of the application and determine optimal paths in intermodal networks with respect to a predetermined set of users’ preferences. The data requirements, the structure and the architecture of the application, the required technologies, the underpinning mathematical context of the proposed algorithms, and the performance of this application in random networks are also presented.

Energy efficient data access and storage through HW/SW co-design

Massive energy consumption has become a major factor for the design and implementation of datacenters. This has led to numerous academic and industrial efforts to improve the energy efficiency of datacenter infrastructures. As a result, in state-of-the-art datacenter facilities, over 80% of power is now consumed by servers themselves. Historically, the processor has dominated energy consumption in the server. However, as processors have become more energy efficient, their contribution has been decreasing. On the contrary, energy consumed by data accesses and storage is growing, since multi- and many-core severs are requiring increased main memory bandwidth/capacity, large register file and large-scale storage system. Accordingly, energy consumed by data accesses and storage approaching or even surpassing that consumed by processors in many servers. For example, it has been reported that main memory contributes to as much as 40-46% of total energy consumption in server applications. In this talk, we present our continuing efforts to improve the energy efficiency of data accesses and storage. We study on a series of approaches with hardware-software cooperation to save energy consumption of on-chip memory, register file, main memory and storage devices for embedded systems, multi- and many-core servers, respectively. Experiments with a large set of workloads show the accuracy of our analytical models and the effectiveness of our optimizations.

A Case for Specialized Processors for Scale-Out Workloads

Emerging scale-out workloads need extensive amounts of computational resources. However, data centers using modern server hardware face physical constraints in space and power, limiting further expansion and requiring improvements in the computational density per server and in the per-operation energy. Continuing to improve the computational resources of the cloud while staying within physical constraints mandates optimizing server efficiency. In this work, we demonstrate that modern server processors are highly inefficient for running cloud workloads. To address this problem, we investigate the microarchitectural behavior of scale-out workloads and present opportunities to enable specialized processor designs that closely match the needs of the cloud.

Optimal Power Allocation and Load Distribution for Multiple Heterogeneous Multicore Server Processors across Clouds and Data Centers

For multiple heterogeneous multicore server processors across clouds and data centers, the aggregated performance of the cloud of clouds can be optimized by load distribution and balancing. Energy efficiency is one of the most important issues for large-scale server systems in current and future data centers. The multicore processor technology provides new levels of performance and energy efficiency. The present paper aims to develop power and performance constrained load distribution methods for cloud computing in current and future large-scale data centers. In particular, we address the problem of optimal power allocation and load distribution for multiple heterogeneous multicore server processors across clouds and data centers. Our strategy is to formulate optimal power allocation and load distribution for multiple servers in a cloud of clouds as optimization problems, i.e., power constrained performance optimization and performance constrained power optimization. Our research problems in large-scale data centers are well-defined multivariable optimization problems, which explore the power-performance tradeoff by fixing one factor and minimizing the other, from the perspective of optimal load distribution. It is clear that such power and performance optimization is important for a cloud computing provider to efficiently utilize all the available resources. We model a multicore server processor as a queuing system with multiple servers. Our optimization problems are solved for two different models of core speed, where one model assumes that a core runs at zero speed when it is idle, and the other model assumes that a core runs at a constant speed. Our results in this paper provide new theoretical insights into power management and performance optimization in data centers.

Tibidabo : Making the case for an ARM-based HPC system

It is widely accepted that future HPC systems will be limited by their power consumption. Current HPC systems are built from commodity server processors, designed over years to achieve maximum performance, with energy efficiency being an after-thought. In this paper we advocate a different approach: building HPC systems from low-power embedded and mobile technology parts, over time designed for maximum energy efficiency, which now show promise for competitive performance.We introduce the architecture of Tibidabo, the first large-scale HPC cluster built from ARM multicore chips, and a detailed performance and energy efficiency evaluation. We present the lessons learned for the design and improvement in energy efficiency of future HPC systems based on such low-power cores. Based on our experience with the prototype, we perform simulations to show that a theoretical cluster of 16-core ARM Cortex-A15 chips would increase the energy efficiency of our cluster by 8.7×, reaching an energy efficiency of 1046 MFLOPS/W.

Virtual Batching: Request Batching for Server Energy Conservation in Virtualized Data Centers

Many power management strategies have been proposed for enterprise servers based on dynamic voltage and frequency scaling (DVFS), but those solutions cannot further reduce the energy consumption of a server when the server processor is already at the lowest DVFS level and the server utilization is still low (e.g., 10 percent or lower). To achieve improved energy efficiency, request batching can be conducted to group received requests into batches and put the processor into sleep between the batches. However, it is challenging to perform request batching on a virtualized server because different virtual machines on the same server may have different workload intensities. Hence, putting the shared processor into sleep may severely impact the application performance of all the virtual machines. This paper proposes Virtual Batching, a novel request batching solution for virtualized servers with primarily light workloads. Our solution dynamically allocates CPU resources such that all the virtual machines can have approximately the same performance level relative to their allowed peak values. Based on this uniform level, Virtual Batching determines the time length for periodically batching incoming requests and putting the processor into sleep. When the workload intensity changes from light to moderate, request batching is automatically switched to DVFS to increase processor frequency for performance guarantees. Virtual Batching is also extended to integrate with server consolidation for maximized energy conservation with performance guarantees for virtualized data centers. Empirical results based on a hardware testbed and real trace files show that Virtual Batching can achieve the desired performance with more energy conservation than several well-designed baselines, e.g., 63 percent more, on average, than a solution based on DVFS only.

SMT Switch: Software Mechanisms for Power Shifting

Simultaneous multithreading (SMT) as a processor design to achieve higher levels of system and application throughput is a well-accepted and deployed technique in most desktop and server processors. We study the power implications of varying SMT levels i.e., thread counts per core for various multi-threaded applications on a real SMT multicore platform, and introduce a novel software mechanism of changing SMT level of a core to tune platform power. Power-shifting policies by varying per core SMT levels for performance benefits within a power cap are introduced. Projected power savings (of 15%) for a streaming parallel benchmark can be attained using SMT-level power shifting mechanisms.

Die-stacked DRAM caches for servers

Recent research advocates using large die-stacked DRAM caches to break the memory bandwidth wall. Existing DRAM cache designs fall into one of two categories --- block-based and page-based. The former organize data in conventional blocks (e.g., 64B), ensuring low off-chip bandwidth utilization, but co-locate tags and data in the stacked DRAM, incurring high lookup latency. Furthermore, such designs suffer from low hit ratios due to poor temporal locality. In contrast, page-based caches, which manage data at larger granularity (e.g., 4KB pages), allow for reduced tag array overhead and fast lookup, and leverage high spatial locality at the cost of moving large amounts of data on and off the chip. This paper introduces Footprint Cache, an efficient die-stacked DRAM cache design for server processors. Footprint Cache allocates data at the granularity of pages, but identifies and fetches only those blocks within a page that will be touched during the page's residency in the cache --- i.e., the page's footprint. In doing so, Footprint Cache eliminates the excessive off-chip traffic associated with page-based designs, while preserving their high hit ratio, small tag array overhead, and low lookup latency. Cycle-accurate simulation results of a 16-core server with up to 512MB Footprint Cache indicate a 57% performance improvement over a baseline chip without a die-stacked cache. Compared to a state-of-the-art block-based design, our design improves performance by 13% while reducing dynamic energy of stacked DRAM by 24%.

Quantifying the Mismatch between Emerging Scale-Out Applications and Modern Processors

Emerging scale-out workloads require extensive amounts of computational resources. However, data centers using modern server hardware face physical constraints in space and power, limiting further expansion and calling for improvements in the computational density per server and in the per-operation energy. Continuing to improve the computational resources of the cloud while staying within physical constraints mandates optimizing server efficiency to ensure that server hardware closely matches the needs of scale-out workloads. In this work, we introduce CloudSuite, a benchmark suite of emerging scale-out workloads. We use performance counters on modern servers to study scale-out workloads, finding that today’s predominant processor microarchitecture is inefficient for running these workloads. We find that inefficiency comes from the mismatch between the workload needs and modern processors, particularly in the organization of instruction and data memory systems and the processor core microarchitecture. Moreover, while today’s predominant microarchitecture is inefficient when executing scale-out workloads, we find that continuing the current trends will further exacerbate the inefficiency in the future. In this work, we identify the key microarchitectural needs of scale-out workloads, calling for a change in the trajectory of server processors that would lead to improved computational density and power efficiency in data centers.

A Parallel Grad-Shafranov Solver for Real-Time Control of Tokamak Plasmas

To achieve real-time control of fusion plasmas, the flux distribution and derived quantities have to be calculated within the time of the machine control cycle, which in the case of the ASDEX-Upgrade experiment can be as small as 1 ms. To this end we have developed a fast numerical solver for the Grad-Shafranov equation, which allows exploitation of the parallel capabilities of modern multicore processors. Our implementation, termed GPEC (Garching parallel equilibrium code), is based entirely on open-source software components. For a numerical grid of size 32 × 64, our new code requires only 0.04 ms (0.11 ms for 64 × 128) for a single call of the Grad-Shafranov solver using a standard Intel Xeon quad-core CPU (3.2 GHz). We also show the first GPEC benchmark results obtained on the Intel Sandy Bridge eight-core server processor and demonstrate the relevance of the new solver for application in plasma equilibrium codes.

Scale-out processors

Scale-out datacenters mandate high per-server throughput to get the maximum benefit from the large TCO investment. Emerging applications (e.g., data serving and web search) that run in these datacenters operate on vast datasets that are not accommodated by on-die caches of existing server chips. Large caches reduce the die area available for cores and lower performance through long access latency when instructions are fetched. Performance on scale-out workloads is maximized through a modestly-sized last-level cache that captures the instruction footprint at the lowest possible access latency. In this work, we introduce a methodology for designing scalable and efficient scale-out server processors. Based on a metric of performance-density, we facilitate the design of optimal multi-core configurations, called pods. Each pod is a complete server that tightly couples a number of cores to a small last-level cache using a fast interconnect. Replicating the pod to fill the die area yields processors which have optimal performance density, leading to maximum per-chip throughput. Moreover, as each pod is a stand-alone server, scale-out processors avoid the expense of global (i.e., inter-pod) interconnect and coherence. These features synergistically maximize throughput, lower design complexity, and improve technology scalability. In 20nm technology, scale-out chips improve throughput by 5x-6.5x over conventional and by 1.6x-1.9x over emerging tiled organizations.

Optimizing Data-Center TCO with Scale-Out Processors

Performance and total cost of ownership (TCO) are key optimization metrics in large-scale data centers. According to these metrics, data centers designed with conventional server processors are inefficient. Recently introduced processors based on low-power cores can improve both throughput and energy efficiency compared to conventional server chips. However, a specialized Scale-Out Processor (SOP) architecture maximizes on-chip computing density to deliver the highest performance per TCO and performance per watt at the data-center level.

The high stakes of low power

There are two giants in the computer processor industry. One is Intel, which builds most of the processors in today's PCs and servers. The other is ARM Holdings, in Cambridge, England, which thanks to its vast ecosystem of partners has established near-complete dominance of the market for the core logic inside smartphones and tablets.