Chip Multiprocessor System Research Articles

Run-time adaptive data page mapping: A Comparison with 3D-stacked DRAM cache

In the current chip-multiprocessor era, 3D-stacked DRAM became an attractive alternative to mitigate the DRAM bandwidth wall problem. In a chip-multiprocessor, the 3D-stacked DRAM is architect either (a) to cache both local and remote data or (b) to cache only the local data. Caching only local data into the 3D-stacked DRAM enforces the chip-multiprocessors to suffer inter-node latency overhead while accessing remote data. However, caching both local and remote data onto the 3D-stacked DRAM requires a large coherence directory (tens of MBs) to ensure correctness. In this paper, we consider a 3D-stacked DRAM based chip-multiprocessor and perform a comparative study between (a) high level adaptive run-time data page mapping onto DRAM with an auxiliary small SRAM buffer as a performance booster, and (b) DRAM used as coherent cache. Our experiment on a 64 core chip-multiprocessor system with 4GB of 3D-stacked DRAM shows that our adaptive run-time data page mapping on DRAM along with an SRAM buffer outperforms the base-case (where DRAM caches only local data) by an average of 48%. Moreover, our method shows a performance improvement by an average of 40% when compared with a recent state-of-art work (where DRAM caches both local and remote data).

Dynamic Data Allocation and Task Scheduling on Multiprocessor Systems With NVM-Based SPM

Low-power and short-latency memory access is critical to the performance of chip multiprocessor (CMP) system devices, especially to bridge the performance gap between memory and CPU. Together with increased demand for low-energy consumption and high-speed memory, scratch-pad memory (SPM) has been widely adopted in multiprocessor systems. In this paper, we employ a hybrid SPM, composed of a static random-access memory and a nonvolatile memory (NVM), to replace the cache in CMP. However, there are several challenges related to the CMP that need to be addressed, including how to dynamically assign processors to application tasks and dynamically allocate data to memories. To solve these problems based on this architecture, we propose a novel dynamic data allocation and task scheduling algorithm, i.e., dynamic greedy data allocation and task scheduling (DGDATS). Experiments on DSP benchmarks demonstrate the effectiveness and efficiency of our proposed algorithms; namely, our proposed algorithm can generate a highly efficient dynamic data allocation and task scheduling approach to minimize the total execution cost and produce the least amount of write operations on NVMs. Our extensive simulation study demonstrates that our proposed algorithm exhibits an excellent performance compared with the heuristic allocation (HA) and adaptive genetic algorithm for data allocation (AGADA) algorithms. Based on the CMP systems with hybrid SPMs, DGDATS reduces the total execution cost by 22.18% and 51.37% compared with those of the HA and AGADA algorithms, respectively. Additionally, the average number of write operations on NVM is 19.82% lower than that of HA.

Dynamic energy management for chip multi-processors under performance constraints

We introduce a novel algorithm for dynamic energy management (DEM) under performance constraints in chip multi-processors (CMPs). Using the novel concept of delayed instructions count, performance loss estimations are calculated at the end of each control period for each core. In addition, a Kalman filtering based approach is employed to predict workload in the next control period for which voltage-frequency pairs must be selected. This selection is done with a novel dynamic voltage and frequency scaling (DVFS) algorithm whose objective is to reduce energy consumption but without degrading performance beyond the user set threshold. Using our customized Sniper based CMP system simulation framework, we demonstrate the effectiveness of the proposed algorithm for a variety of benchmarks for 16 core and 64 core network-on-chip based CMP architectures. Simulation results show consistent energy savings across the board. We present our work as an investigation of the tradeoff between the achievable energy reduction via DVFS when predictions are done using the effective Kalman filter for different performance penalty thresholds.

Path Selection for Real-Time Communication on Priority-Aware NoCs

This work investigates selecting paths for communication flows when deploying a hard real-time application on a chip-multiprocessor system. This chip-multiprocessor system uses a priority-aware real-time network-on-chip interconnect between the processors. Given a mapping of the computation tasks onto the chip-multiprocessor, the problem we address in this work is to discover paths the communication flows take such that hard real-time deadlines of flows are met. Furthermore, we must ensure that deadlines are met even in the presence of direct and indirect interference from other flows sharing network links on the path. To achieve this, our algorithm utilizes a stage-level analysis for real-time communication to determine the impact of a network link being used by a flow, and its effect on other flows sharing the link. The path selection algorithm uses heuristics such as selecting links with least interference, and considering lower-priority flows when dedicating links to paths of higher-priority flows since an optimal one is intractable. The algorithm also considers constraints on the number of virtual channels at each router port in the network. The statistically significant experimental results show an improvement in schedulability by 5% and 12% over existing path selection algorithms such as Minimum Interference Routing and Widest Shortest Path algorithms, respectively. We also present a set-top box case study to further illustrate the benefits of using the proposed algorithm.

The efficiency of buffer and buffer-less data-flow control schemes for congestion avoidance in Networks on Chip

The design of efficient architectures for communication in on chip multiprocessors system involves many challenges regarding the internal router functions used in Network on Chip (NoC) infrastructure. The on-chip router should be designed to provide per-flit processing with enhanced granularity. In fact, the quality of service experienced at the application level depends on the capabilities of the router to avoid congestion and to ensure efficient data-flow control. Consequently, an enhanced router architecture is needed to achieve the requested QoS.This paper proposes an internal router architecture, for on chip communication, implementing flow-control mechanism for congestion avoidance with QoS consideration. It describes the internal functions of this router for optimal output flit scheduling and its capability to apply per-class service for inbound flows. The paper focuses mainly on the description and performance analysis of two proposed schemes for data flow control that can be used with the proposed router architecture. The results shown in this paper prove that the application of these proposed schemes in NoC achieves an interesting enhancement in the measured end to end QoS. We carried out an extensive comparison of the proposed solutions with the existing schemes published in the literature to show that the proposed solution outperforms these, maintaining an interesting tradeoff with the hardware characteristics when designed with 45nm integration technology.

Phase-Change Memory Optimization for Green Cloud with Genetic Algorithm

Green cloud is an emerging new technology in the computing world in which memory is a critical component. Phase-change memory (PCM) is one of the most promising alternative techniques to the dynamic random access memory (DRAM) that faces the scalability wall. Recent research has been focusing on the multi-level cell (MLC) of PCM. By precisely arranging multiple levels of resistance inside a PCM cell, more than one bit of data can be stored in one single PCM cell. However, the MLC PCM suffers from the degradation of performance compared to the single-level cell (SLC) PCM, due to the longer memory access time. In this paper, we present a genetic-based optimization algorithm for chip multiprocessor (CMP) equipped with PCM memory in green clouds. The proposed genetic-based algorithm not only schedules and assigns tasks to cores in the CMP system, but also provides a PCM MLC configuration that balances the PCM memory performance as well as the efficiency. The experimental results show that our genetic-based algorithm can significantly reduce the maximum memory usage by 76.8 percent comparing with the uniform SLC configuration, and improve the efficiency of memory usage by 127 percent comparing with the uniform 4 bits/cell MLC configuration. Moreover, the performance of the system is also improved by 24.5 percent comparing with the uniform 4 bits/cell MLC configuration in terms of total execution time.

Network aware performance evaluation of prefetching techniques in CMPs

This study focuses on the importance of quantifying the effect of prefetching on the interconnection network of a multiprocessor chip. This kind of microarchitectural effects are often quantified using simulators. However, if prefetching traffic in a CMP (Chip MultiProcessor) system is to be accurately evaluated, simulators should simulate not only the memory hierarchy module and the multicore system, but also the network-on-chip. Unfortunately, no open-source simulator is able to evaluate all these elements at the same time. This paper describes how to develop a prefetching module for the gem5 CMP simulator and how to integrate this into the Ruby memory system.Moreover, by using the infrastructure developed in this study, this paper shows the importance of taking the network effect in prefetching-related studies into account, in order for accurate results to be obtained: not doing so may lead to mistaken conclusions. For this purpose, we have carried out a detailed analysis of the behavior of three different prefetching engines, providing not only the typical statistics for instructions per cycle and the miss rate, but also specific network and prefetching statistics.

A circuit-architecture co-optimization framework for exploring nonvolatile memory hierarchies

Many new memory technologies are available for building future energy-efficient memory hierarchies. It is necessary to have a framework that can quickly find the optimal memory technology at each hierarchy level. In this work, we first build a circuit-architecture joint design space exploration framework by combining RC circuit analysis and Artificial Neural Network (ANN)-based performance modeling. Then, we use this framework to evaluate some emerging nonvolatile memory hierarchies. We demonstrate that a Resistive RAM (ReRAM)-based cache hierarchy on an 8-core Chip-Multiprocessor (CMP) system can achieve a 24% Energy Delay Product (EDP) improvement and a 36% Energy Delay Area Product (EDAP) improvement compared to a conventional hierarchy with SRAM on-chip caches and DRAM main memory.

Study of Various Factors Affecting Performance of Multi-Core Processors

Advances in Integrated Circuit processing allow for more microprocessor design options. As Chip Multiprocessor system (CMP) become the predominant topology for leading microprocessors, critical components of the system are now integrated on a single chip. This enables sharing of computation resources that was not previously possible. In addition the virtualization of these computation resources exposes the system to a mix of diverse and competing workloads. On chip Cache memory is a resource of primary concern as it can be dominant in controlling overall throughput. This Paper presents analysis of various parameters affecting the performance of Multi-core Architectures like varying the number of cores, changes L2 cache size, further we have varied directory size from 64 to 2048 entries on a 4 node, 8 node 16 node and 64 node Chip multiprocessor which in turn presents an open area of research on multicore processors with private/shared last level cache as the future trend seems to be towards tiled architecture executing multiple parallel applications with optimized silicon area utilization and excellent performance.

Reducing memory access latency with asymmetric DRAM bank organizations

DRAM has been a de facto standard for main memory, and advances in process technology have led to a rapid increase in its capacity and bandwidth. In contrast, its random access latency has remained relatively stagnant, as it is still around 100 CPU clock cycles. Modern computer systems rely on caches or other latency tolerance techniques to lower the average access latency. However, not all applications have ample parallelism or locality that would help hide or reduce the latency. Moreover, applications' demands for memory space continue to grow, while the capacity gap between last-level caches and main memory is unlikely to shrink. Consequently, reducing the main-memory latency is important for application performance. Unfortunately, previous proposals have not adequately addressed this problem, as they have focused only on improving the bandwidth and capacity or reduced the latency at the cost of significant area overhead. We propose asymmetric DRAM bank organizations to reduce the average main-memory access latency. We first analyze the access and cycle times of a modern DRAM device to identify key delay components for latency reduction. Then we reorganize a subset of DRAM banks to reduce their access and cycle times by half with low area overhead. By synergistically combining these reorganized DRAM banks with support for non-uniform bank accesses, we introduce a novel DRAM bank organization with center high-aspect-ratio mats called CHARM. Experiments on a simulated chip-multiprocessor system show that CHARM improves both the instructions per cycle and system-wide energy-delay product up to 21% and 32%, respectively, with only a 3% increase in die area.

TLB Improvements for Chip Multiprocessors

Translation Lookaside Buffers (TLBs) are critical to overall system performance. Much past research has addressed uniprocessor TLBs, lowering access times and miss rates. However, as Chip MultiProcessors (CMPs) become ubiquitous, TLB design and performance must be reevaluated. Our article begins by performing a thorough TLB performance evaluation of sequential and parallel benchmarks running on a real-world, modern CMP system using hardware performance counters. This analysis demonstrates the need for further improvement of TLB hit rates for both classes of application, and it also points out that the data TLB has a significantly higher miss rate than the instruction TLB in both cases. In response to the characterization data, we propose and evaluate both Inter-Core Cooperative (ICC) TLB prefetchers and Shared Last-Level (SLL) TLBs as alternatives to the commercial norm of private, per-core L2 TLBs. ICC prefetchers eliminate 19% to 90% of Data TLB (D-TLB) misses across parallel workloads while requiring only modest changes in hardware. SLL TLBs eliminate 7% to 79% of D-TLB misses for parallel workloads and 35% to 95% of D-TLB misses for multiprogrammed sequential workloads. This corresponds to 27% and 21% increases in hit rates as compared to private, per-core L2 TLBs, respectively, and is achieved this using even more modest hardware requirements. Because of their benefits for parallel applications, their applicability to sequential workloads, and their readily implementable hardware, SLL TLBs and ICC TLB prefetchers hold great promise for CMPs.

Three-phase time-aware energy minimization with DVFS and unrolling for Chip Multiprocessors

Energy consumption has been one of the most critical issues in the Chip Multiprocessor (CMP). Using the Dynamic Voltage and Frequency Scaling (DVFS), a CMP system can achieve a balance between the performance and the energy-efficiency. In this paper, we propose a three-phase discrete DVFS algorithm for a CMP system dedicated to applications where the period of the applications’ task graph is smaller than the deadline of tasks. In these applications, multiple task graphs are unrolled and then concatenated together to form a new task graph. The proposed DVFS algorithm is applied to the newly formed task graph to stretch tasks’ execution time, lower operating frequencies of processors and achieve the system power efficiency. Experimental results show that the proposed algorithm reduces the energy dissipation by 25% on average, compared to previous DVFS approaches.

Improving System Energy Efficiency with Memory Rank Subsetting

VLSI process technology scaling has enabled dramatic improvements in the capacity and peak bandwidth of DRAM devices. However, current standard DDR x DIMM memory interfaces are not well tailored to achieve high energy efficiency and performance in modern chip-multiprocessor-based computer systems. Their suboptimal performance and energy inefficiency can have a significant impact on system-wide efficiency since much of the system power dissipation is due to memory power. New memory interfaces, better suited for future many-core systems, are needed. In response, there are recent proposals to enhance the energy efficiency of main-memory systems by dividing a memory rank into subsets, and making a subset rather than a whole rank serve a memory request. We holistically assess the effectiveness of rank subsetting from system-wide performance, energy-efficiency, and reliability perspectives. We identify the impact of rank subsetting on memory power and processor performance analytically, compare two promising rank-subsetting proposals, Multicore DIMM and mini-rank, and verify our analysis by simulating a chip-multiprocessor system using multithreaded and consolidated workloads. We extend the design of Multicore DIMM for high-reliability systems and show that compared with conventional chipkill approaches, rank subsetting can lead to much higher system-level energy efficiency and performance at the cost of additional DRAM devices. This holistic assessment shows that rank subsetting offers compelling alternatives to existing processor-memory interfaces for future DDR systems.

Parallel Simulations of Dynamic Earthquake Rupture along Geometrically Complex Faults on CMP Systems

Chip multiprocessors (CMP) are widely used for high performance computing and are being configured in a hierarchical manner to compose a CMP compute node in a CMP system. Such a CMP system provides a natural programming paradigm for hybrid MPI/OpenMP applications. In this paper, we use OpenMP to parallelize a sequential earthquake simulation code for modeling spontaneous earthquake rupture along geometrically complex faults on two CMP systems, IBM POWER5+ system and SUN Opteron server. The experimental results indicate that the OpenMP implementation has the accurate output results and the good scalability on the two CMP systems. We apply the optimization techniques such as large page and processor binding to the OpenMP implementation to achieve up to 7.05% performance improvement on the CMP systems without any code modification. Further, we illustrate an element-based partitioning scheme for explicit finite element methods. Based on the partitioning scheme and what we learn from the OpenMP implementation, we discuss how efficiently to use hybrid MPI/OpenMP to parallelize the sequential earthquake rupture simulation code in order to not only achieve multiple levels of parallelism of the code but also to reduce the communication overhead of MPI within a CMP node by taking advantage of the shared address space and on-chip high inter-core bandwidth and low inter-core latency. Our initial experimental results indicate that the hybrid MPI/OpenMP implementation obtains the accurate output results and has good scalability on CMP systems.

Event-driven configuration of a neural network CMP system over an homogeneous interconnect fabric

Configuring a million-core parallel system at boot time is a difficult process when the system has neither specialised hardware support for the configuration process nor a preconfigured default state that puts it in operating condition. The architecture of SpiNNaker, a parallel chip multiprocessor (CMP) system for neural network simulation, is in this class. To function as a universal neural chip, SpiNNaker uses an event-driven model with complete system virtualisation so that all components are generic and identical. Where most large CMP systems feature a sideband network to complete the boot process, SpiNNaker has a single homogeneous network interconnect for both application inter-processor communications and system control functions. This network improves fault tolerance and makes it easier to support dynamic run-time reconfiguration, however, it requires a boot process compatible with the application’s communications model. Here, we present such a boot loader, capable of bringing a generic, initially unconfigured parallel system into a working configuration. Since SpiNNaker uses event-driven asynchronous communications throughout, the loader operates with purely local control: there is no global synchronisation, state information, or transition sequence. A novel two-stage “unfolding” boot-up process efficiently configures the SpiNNaker hardware and loads the application using a high-speed flood-fill technique with support for run-time reconfiguration. SystemC simulation of a multi-CMP SpiNNaker system indicates an error-free CMP configuration time of ∼1.37 ms, while a high-level simulation of a full-scale system (64 K CMPs) indicates a mean application-loading time of ∼20 ms (for a 100 KB application), which is virtually independent of the size of the system. Further hardware-level Verilog simulation verified the cycle-accurate functionality of CMP configuration. The complete process illustrates a useful method for configuring large-scale event-driven parallel systems without having to provide dedicated hardware boot support or rely on system state assumptions.

Location Cache Design and Performance Analysis for Chip Multiprocessors

Recent research at Intel suggests that chips with hundreds of processor cores are possible in the not-so-distant future. As the number of cores grows, so does the size of the cache systems required to allow them to operate efficiently. Caches have grown to consume a significant percentage of the power utilized by a processor. In this research, we extend the concept of location cache to support chip multiprocessors (CMPs) systems in combination with low-power L2 caches based upon the gated-ground technique. The combination of these two techniques allows for reductions in both dynamic and leakage power consumption. In this paper, we will present an analysis of the power savings provided by utilizing location caches in a CMP system. The performance of the cache system is evaluated by extending the capability of CACTI and Simics using the SPLASH-2 and ALPBench benchmark suites. These simulation results demonstrate that the utilization of location caches in CMP systems is capable of saving a significant amount of power over equivalent CMP systems that lack location caches.

Putting Faulty Cores to Work

Necromancer, a robust and heterogeneous core coupling execution scheme, exploits a functionally dead core to improve system throughput by supplying hints regarding high-level program behavior. Necromancer partitions a chip multiprocessor system's cores into multiple groups, each of which shares a lightweight core that can be substantially accelerated using execution hints from the faulty core.

Design and Analysis of Location Caches in a NoC-Based Chip Multiprocessor System

This research is focused on designing location caches for a NoC-based NUCA (non-uniform cache architecture) system with multiple cores, in combination with a low-leakage L2 cache based on the gated-ground technique. This system architecture helps reduce the power consumption of the L2 cache along with the performance benefit of the on-chip network. The features of CACTI and GEMS are extended to support complete power and performance simulation of the system. The full-system simulation is performed on scientific and multimedia workloads to characterize the NoC-based system. Analysis of the power and performance of the proposed CMP cache system is presented in comparison with the traditional cache structure in different configurations. Simulation results show that the NoC-based system with location caches significantly saves the energy of the cache system over the traditional bus-based system in any configuration and also the NoC-based system without location caches. The system also provides better performance compared to a bus-based system, emphasizing the need to shift to a network-based cache interconnect design which can scale to a large number of cores.

Understanding sources of inefficiency in general-purpose chips

Due to their high volume, general-purpose processors, and now chip multiprocessors (CMPs), are much more cost effective than ASICs, but lag significantly in terms of performance and energy efficiency. This paper explores the sources of these performance and energy overheads in general-purpose processing systems by quantifying the overheads of a 720p HD H.264 encoder running on a general-purpose CMP system. It then explores methods to eliminate these overheads by transforming the CPU into a specialized system for H.264 encoding. We evaluate the gains from customizations useful to broad classes of algorithms, such as SIMD units, as well as those specific to particular computation, such as customized storage and functional units. The ASIC is 500x more energy efficient than our original four-processor CMP. Broadly applicable optimizations improve performance by 10x and energy by 7x. However, the very low energy costs of actual core ops (100s fJ in 90nm) mean that over 90% of the energy used in these solutions is still "overhead". Achieving ASIC-like performance and efficiency requires algorithm-specific optimizations. For each sub-algorithm of H.264, we create a large, specialized functional unit that is capable of executing 100s of operations per instruction. This improves performance and energy by an additional 25x and the final customized CMP matches an ASIC solution's performance within 3x of its energy and within comparable area.

Fairness via source throttling

Cores in a chip-multiprocessor (CMP) system share multiple hardware resources in the memory subsystem. If resource sharing is unfair, some applications can be delayed significantly while others are unfairly prioritized. Previous research proposed separate fairness mechanisms in each individual resource. Such resource-based fairness mechanisms implemented independently in each resource can make contradictory decisions, leading to low fairness and loss of performance. Therefore, a coordinated mechanism that provides fairness in the entire shared memory system is desirable. This paper proposes a new approach that provides fairness in the entire shared memory system , thereby eliminating the need for and complexity of developing fairness mechanisms for each individual resource. Our technique, Fairness via Source Throttling (FST), estimates the unfairness in the entire shared memory system. If the estimated unfairness is above a threshold set by system software, FST throttles down cores causing unfairness by limiting the number of requests they can inject into the system and the frequency at which they do. As such, our source-based fairness control ensures fairness decisions are made in tandem in the entire memory system. FST also enforces thread priorities/weights, and enables system software to enforce different fairness objectives and fairness-performance tradeoffs in the memory system. Our evaluations show that FST provides the best system fairness and performance compared to four systems with no fairness control and with state-of-the-art fairness mechanisms implemented in both shared caches and memory controllers.