Large-scale Chip Multiprocessors Research Articles

Tag-Sharer-Fusion Directory: A Scalable Coherence Directory With Flexible Entry Formats

In large-scale chip multiprocessors (CMPs), the scalability of a coherence directory becomes more important as the number of cores increases. However, previously proposed scalable coherence directories typically reduce the directory storage overhead at the cost of one or more aspects of performance, accuracy, and complexity. In this article, we propose the tag-sharer-fusion (TSF) directory, a scalable coherence directory with low hardware complexity, as well as with high performance and accuracy. Each directory entry has just enough bits to store a single sharer pointer and is divided into two primary formats: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tag</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sharer</i> , where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sharer</i> entries store sharers but not tags. Each private block is tracked by a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tag</i> entry, and each shared block is tracked by a combination of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tag</i> entry and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sharer</i> entry in the same set. Simulation of a 128-core chip-multiprocessor with the PARSEC and SPLASH-2x benchmarks shows that the TSF directory requires only a quarter of the area of a non-scalable full-map sparse directory to achieve similar performance and network traffic, both with an average overhead within 1%. The TSF directory outperforms the state-of-the-art Pool and way-combining directory proposals in terms of storage overhead, performance, and network traffic.

Toward a High-Performance and Low-Loss Clos–Benes-Based Optical Network-on-Chip Architecture

As chip multiprocessors (CMPs) keep growing in capability, on-chip communication efficiency is crucial to the overall performance. However, on-chip networks based on electronic switches suffer from excessive power consumption and limited performance. In order to take advantages of optical interconnect, we propose an optimized design toward a high-performance and low-loss Clos–Benes-based hierarchical optical network-on-chip (NoC) for large-scale CMPs. We propose several key techniques, including a loss-aware adaptive (LAA) routing for intraswitch Benes network, a priority-based round-Robin virtual output queue selection and a $Q$ -learning-based heuristic routing for interswitch Clos network, and a local transfer link (LTL) technique to improve the traffic locality. A case study on a 256-core CMP under uniform traffic shows that the network throughput is increased by 346.7%, 61%, and 12.9%, respectively, than the mesh, fattree, and the traditional generic Clos–Benes optical NoC. On average of a set of real applications, the application end-to-end delay is reduced by 47.6%, 28.2%, and 19.4%, respectively, than the mesh, fattree, and the traditional generic Clos–Benes network. Meanwhile, the average optical power loss is decreased by 11.8% and 49.9%, respectively, as compared to the mesh and fattree. As compared to a baseline Clos–Benes network, the use of LAA routing together with the LTL could reduce the average optical power loss by 28.4%.

OS Level Power Optimization in LCMP

With the increasing levels of transistor count and clock rate of microprocessors there is a significant increase in power dissipation. Reducing power consumption in both high power consumption and high performance has developed into one of the main target in designing a system for various devices. As the chip multiprocessor (CMP) are integrating more cores on the die, it will leads to the extent of Large scale CMP (LCMP) architectures with potentially hundreds of thread on the die and thousands of cores. Therefore, we proposed an approach of OS level power optimization in LCMP to optimize the heat dissipation rate and increase computing power under some considerations. To satisfy the main goal of our work, the heat dissipation should be optimizing with increase in computing power. The approach of optimizing the heat dissipation is done at the synthesis level. There are three approaches for modifying the synthetic benchmark: Singly Synthesis, Hierarchical Synthesis and Group Synthesis. The result is that the power dissipation of Group synthesis is equally distributed without giving more loads to only one processor as compared to Hierarchical Synthesis and Singly Synthesis. Therefore, from our result we can conclude that in Group Synthesis power is equally distributed hence heat dissipation is optimized. The future work will be to further optimize the result of the Synthesis level using thread migration. Thread Migration can increase the system throughput; it relies on multiple cores that vary in performance capabilities.

Broadcast-Enabled Massive Multicore Architectures: A Wireless RF Approach

Broadcast traditionally has been regarded as a prohibitive communication transaction in multiprocessor environments. Nowadays, such a constraint largely drives the design of architectures and algorithms all-pervasive in diverse computing domains, directly and indirectly leading to diminishing performance returns as the many-core era is approaching. Novel interconnect technologies could help revert this trend by offering, among others, improved broadcast support, even in large-scale chip multiprocessors. This article outlines the prospects of wireless on-chip communication technologies pointing toward low-latency (a few cycles) and energy-efficient broadcast (a few picojoules per bit). It also discusses the challenges and potential impact of adopting these technologies as key enablers of unconventional hardware architectures and algorithmic approaches, in the pathway of significantly improving the performance, energy efficiency, scalability, and programmability of many-core chips.

Hybrid circuit-switched network for on-chip communication in large-scale chip-multiprocessors

Large-scale chip-multiprocessors (CMPs) need a scalable communication structure characterized by low cost, low power, and high performance to meet their on-chip communication requirements. This paper presents a hybrid circuit-switched (HCS) network for on-chip communication in the large-scale CMPs. The HCS network, which is Advanced Microcontroller Bus Architecture (AMBA) compatible, is composed of bufferless switches, pipeline channels, and network interfaces. Furthermore, packets are transferred in a hybrid transmission scheme. If a message has only one packet, the transmission scheme for this message is packet switching. Conversely, if a message contains multiple packets, the transmission scheme for this message is circuit switching. We evaluate HCS networks with different channel depths and then compare the HCS network with the Stanford elastic buffer (EB) network. Our results show that the HCS network with two-depth channel requires 83% less power and occupies 32% less area compared with the EB network. Furthermore, under maximum frequency and single traffic, the HCS network with two-depth channel provides 37% lower zero-load latency, 390% higher maximum throughput per unit power, and 19% higher maximum throughput per unit area compared with the EB network.

SUOR

Chip multiprocessor (CMP) is becoming an attractive platform for applications seeking both high performance and high energy efficiency. In large-scale CMPs, the communication efficiency among cores is crucial for the overall system performance and energy consumption. In this article, we propose a ring-based optical network-on-chip, called SUOR, to fulfill the communication requirement of CMPs. SUOR effectively explores the distinctive properties of optical signals and photonic devices, and dynamically partitions each data channel into multiple sections. Each section can be utilized independently to boost performance as well as reduce energy consumption. We develop a set of distributed control protocols and algorithms for SUOR, but physically allocate the corresponding cluster agents close to each other to benefit from the strengths of optical interconnects at long distances as well as electrical interconnects at short distances. Simulation results show that SUOR outperforms the alternative optical networks under a wide range of traffic patterns. For example, compared with MWSR design, SUOR achieves 2.58× throughput as well as saves 64% energy consumption on average in a 256-core CMP. Compared with MWMR design, SUOR achieves 1.52× throughput and reduces 73% energy consumption on average.

PDG_GEN: A Methodology for Fast and Accurate Simulation of On-Chip Networks

With the advent of large scale chip multiprocessors, there is growing interest in the design and analysis of on-chip networks. Full-system simulation is the most accurate way to perform such an analysis, but unfortunately it is very slow and thus limits design space exploration. To overcome this problem researchers frequently use trace-based simulation to study different network topologies and properties, which can be done much faster. Unfortunately, unless the traces that are used include information about dependencies between packets, trace-based simulations can lead one to draw incorrect conclusions about network performance metrics such as average packet latency and overall execution time. The primary contributions of this work are to demonstrate the importance of including dependency information in traces, and to present PDG_GEN, an inference-based technique for identifying and including dependencies in traces. This technique uses traces obtained from multiple full-system simulations of an application of interest to infer dependency information between packets and augment traces with this information. On the SPLASH-2 benchmark suite, PDG_GEN is 2.3 times more accurate at predicting overall execution time and almost 4,000 times more accurate at predicting average packet latency than traditional trace-based methods.

Studying multicore processor scaling via reuse distance analysis

The trend for multicore processors is towards increasing numbers of cores, with 100s of cores-- i.e . large-scale chip multiprocessors (LCMPs)--possible in the future. The key to realizing the potential of LCMPs is the cache hierarchy, so studying how memory performance will scale is crucial. Reuse distance (RD) analysis can help architects do this. In particular, recent work has developed concurrent reuse distance (CRD) and private reuse distance (PRD) profiles to enable analysis of shared and private caches. Also, techniques have been developed to predict profiles across problem size and core count, enabling the analysis of configurations that are too large to simulate. This paper applies RD analysis to study the scalability of multicore cache hierarchies. We present a framework based on CRD and PRD profiles for reasoning about the locality impact of core count and problem scaling. We find interference-based locality degradation is more significant than sharing-based locality degradation. For 256 cores running small problems, the former occurs at small cache sizes, allowing moderate capacity scaling of multicore caches to achieve the same cache performance (MPKI) as a single-core cache. At very large problems, interference-based locality degradation increases significantly in many of our benchmarks. For shared caches, this prevents most of our benchmarks from achieving constant-MPKI scaling within a 256 MB capacity budget; for private caches, all benchmarks cannot achieve constant-MPKI scaling within 256 MB.

Enhancing effective throughput for transmission line-based bus

Main-stream general-purpose microprocessors require a collection of high-performance interconnects to supply the necessary data movement. The trend of continued increase in core count has prompted designs of packet-switched network as a scalable solution for future-generation chips. However, the cost of scalability can be significant and especially hard to justify for smaller-scale chips. In contrast, a circuit-switched bus using transmission lines and corresponding circuits offers lower latencies and much lower energy costs for smaller-scale chips, making it a better choice than a full-blown network-on-chip (NoC) architecture. However, shared-medium designs are perceived as only a niche solution for small- to medium-scale chips. In this paper, we show that there are many low-cost mechanisms to enhance the effective throughput of a bus architecture. When a handful of highly cost-effective techniques are applied, the performance advantage of even the most idealistically configured NoCs becomes vanishingly small. We find transmission line-based buses to be a more compelling interconnect even for large-scale chip-multiprocessors, and thus bring into doubt the centrality of packet switching in future on-chip interconnect.

Scalable and Efficient Fine-Grained Cache Partitioning with Vantage

The Vantage cache-partitioning technique enables configurability and quality-of-service guarantees in large-scale chip multiprocessors with shared caches. Caches can have hundreds of partitions with sizes specified at cache line granularity, while maintaining high associativity and strict isolation among partitions.

Cache-Integrated Network Interfaces: Flexible On-Chip Communication and Synchronization for Large-Scale CMPs

Per-core scratchpad memories (or local stores) allow direct inter-core communication, with latency and energy advantages over coherent cache-based communication, especially as CMP architectures become more distributed. We have designed cache-integrated network interfaces, appropriate for scalable multicores, that combine the best of two worlds – the flexibility of caches and the efficiency of scratchpad memories: on-chip SRAM is configurably shared among caching, scratchpad, and virtualized network interface (NI) functions. This paper presents our architecture, which provides local and remote scratchpad access, to either individual words or multiword blocks through RDMA copy. Furthermore, we introduce event responses, as a technique that enables software configurable communication and synchronization primitives. We present three event response mechanisms that expose NI functionality to software, for multiword transfer initiation, completion notifications for software selected sets of arbitrary size transfers, and multi-party synchronization queues. We implemented these mechanisms in a four-core FPGA prototype, and measure the logic overhead over a cache-only design for basic NI functionality to be less than 20%. We also evaluate the on-chip communication performance on the prototype, as well as the performance of synchronization functions with simulation of CMPs with up to 128 cores. We demonstrate efficient synchronization, low-overhead communication, and amortized-overhead bulk transfers, which allow parallelization gains for fine-grain tasks, and efficient exploitation of the hardware bandwidth.

Predictive analysis of a hydrodynamics application on large-scale CMP clusters

We present the development of a predictive performance model for the high-performance computing code Hydra, a hydrodynamics benchmark developed and maintained by the United Kingdom Atomic Weapons Establishment (AWE). The developed model elucidates the parallel computation of Hydra, with which it is possible to predict its run-time and scaling performance on varying large-scale chip multiprocessor (CMP) clusters. A key feature of the model is its granularity; with the model we are able to separate the contributing costs, including computation, point-to-point communications, collectives, message buffering and message synchronisation. The predictions are validated on two contrasting large-scale HPC systems, an AMD Opteron/InfiniBand cluster and an IBM BlueGene/P, both of which are located at the Lawrence Livermore National Laboratory (LLNL) in the US. We validate the model on up to 2,048 cores, where it achieves a >85% accuracy in weak-scaling studies. We also demonstrate use of the model in exposing the increasing costs of collectives for this application, and also the influence of node density on network accesses, therefore highlighting the impact of machine choice when running this hydrodynamics application at scale.

C-AMTE: A location mechanism for flexible cache management in chip multiprocessors

This paper describes Constrained Associative-Mapping-of-Tracking-Entries (C-AMTE), a scalable mechanism to facilitate flexible and efficient distributed cache management in large-scale chip multiprocessors (CMPs). C-AMTE enables fast locating of cache blocks in CMP cache schemes that employ one-to-one or one-to-many associative mappings. C-AMTE stores in per-core data structures tracking entries to avoid on-chip interconnect traffic outburst or long distance directory lookups. Simulation results using a full system simulator demonstrate that C-AMTE achieves improvement in cache access latency by up to 34.4%, close to that of a perfect location strategy.

An analysis of on-chip interconnection networks for large-scale chip multiprocessors

With the number of cores of chip multiprocessors (CMPs) rapidly growing as technology scales down, connecting the different components of a CMP in a scalable and efficient way becomes increasingly challenging. In this article, we explore the architectural-level implications of interconnection network design for CMPs with up to 128 fine-grain multithreaded cores. We evaluate and compare different network topologies using accurate simulation of the full chip, including the memory hierarchy and interconnect, and using a diverse set of scientific and engineering workloads. We find that the interconnect has a large impact on performance, as it is responsible for 60% to 75% of the miss latency. Latency, and not bandwidth, is the primary performance constraint, since, even with many threads per core and workloads with high miss rates, networks with enough bandwidth can be efficiently implemented for the system scales we consider. From the topologies we study, the flattened butterfly consistently outperforms the mesh and fat tree on all workloads, leading to performance advantages of up to 22%. We also show that considering interconnect and memory hierarchy together when designing large-scale CMPs is crucial, and neglecting either of the two can lead to incorrect conclusions. Finally, the effect of the interconnect on overall performance becomes more important as the number of cores increases, making interconnection choices especially critical when scaling up.

A Dynamic Pressure-Aware Associative Placement Strategy for Large Scale Chip Multiprocessors

This paper describes dynamic pressure-aware associative placement (DPAP), a novel distributed cache management scheme for large-scale chip multiprocessors. Our work is motivated by the large non-uniform distribution of memory accesses across cache sets in different L2 banks. DPAP decouples the physical locations of cache blocks from their addresses for the sake of reducing misses caused by destructive interferences. Temporal pressure at the on-chip last-level cache, is continuously collected at a group (comprised of local cache sets) granularity, and periodically recorded at the memory controller(s) to guide the placement process. An incoming block is consequently placed at a cache group that exhibits the minimum pressure. Simulation results using a full-system simulator demonstrate that DPAP outperforms the baseline shared NUCA scheme by an average of 8.3% and by as much as 18.9% for the benchmark programs we examined. Furthermore, evaluations showed that DPAP outperforms related cache designs.

Mixing It Up

Micro's editor in chief introduces the topics covered by the four articles in this general-interest issue: an interconnection network using highly integrated photonic technology; the ManySim simulation framework for future large-scale chip-multiprocessors; the SimWattch simulation framework, which integrates the Simics functional simulator with the SimpleScalar/Wattch microarchitecture simulators; and self-configuring embedded systems.

Physical simulation for animation and visual effects

We explore the emerging application area of physics-based simulation for computer animation and visual special effects. In particular, we examine its parallelization potential and characterize its behavior on a chip multiprocessor (CMP). Applications in this domain model and simulate natural phenomena, and often direct visual components of motion pictures. We study a set of three workloads that exemplify the span and complexity of physical simulation applications used in a production environment: fluid dynamics, facial animation, and cloth simulation. They are computationally demanding, requiring from a few seconds to several minutes to simulate a single frame; therefore, they can benefit greatly from the acceleration possible with large scale CMPs. Starting with serial versions of these applications, we parallelize code accounting for at least 96% of the serial execution time, targeting a large number of threads.We then study the most expensive modules using a simulated 64-core CMP. For the code representing key modules, we achieve parallel scaling of 45x, 50x, and 30x for fluid, face, and cloth simulations, respectively. The modules have a spectrum of parallel task granularity and locking behavior, and all but one are dominated by loop-level parallelism. Many modules operate on streams of data. In some cases, modules iterate over their data, leading to significant temporal locality. This streaming behavior leads to very high on-die and main memory bandwidth requirements. Finally, most modules have little inter-thread communication since they are data-parallel, but a few require heavy communication between data-parallel operations.

Supporting Microthread Scheduling and Synchronisation in CMPs

Chip multiprocessors (CMPs) hold great promise for achieving scalability in future systems. Microthreaded CMPs add a means of exploiting legacy code in such systems. Using this model, compilers generate parametric concurrency from sequential source code, which can be used to optimise a range of operational parameters such as power and performance over many orders of magnitude, given a scalable implementation. This paper shows scalability in performance, power and most importantly, in silicon implementation, the main contribution of this paper. The microthread model requires dynamic register allocation and a hardware scheduler, which must support hundreds of microthreads per processor. The scheduler must support thread creation, context switching and thread rescheduling on every machine cycle to fully support this model, which is a significant challenge. Scalable implementations of such support structures are given and the feasibility of large-scale CMPs is investigated by giving detailed area estimate of these structures.

Exploring the cache design space for large scale CMPs

With the advent of dual-core chips in the marketplace, small-scale CMP (chip multiprocessor) architectures are becoming commonplace. We expect a continuing trend of increasing the number of cores on a die to maximize the performance/power efficiency of a single chip. We believe an era of large-scale CMPs (LCMPs) with several tens to hundreds of cores is on the way, but as of now architects have little understanding of how best to build a cache hierarchy given such a large number of cores/threads to support. With this in mind, our initial goals are to prune the cache design space for LCMPs by characterizing basic server workload behavior in such an environment.In this paper, we describe the range of methodologies that we are developing to overcome the challenges of exploring the cache design space for LCMP platforms. We then focus on employing a trace-driven approach to characterizing one key server workload (OLTP) in both a homogeneous and a heterogeneous workload environment. We study the effect of increasing threads (from 1 to 128) on a three-level cache hierarchy with emphasis on second and third level caches. We study the effect of varying sizes at these cache levels and show the effects of threads contending for cache space, the effects of prefetching instruction addresses, and the effects of inclusion. We make initial observations and conclusions about the factors on which LCMP cache hierarchy design decisions should be based and discuss future work.