Chip Multiprocessors Research Articles

PR-LRU: partial random LRU technique for performance improvement of last level cache

As chip multiprocessors (CMP) have become eminent in all areas of computing, it is inevitable for the operating system to schedule processes efficiently on different cores. These multi-cores pose different challenges of which shared resource contention is the dominant one, as cores share resources like last level cache (LLC) and main memory. This can lead to poor and unpredictable performance of the threads running on the system. The cache replacement policy of LLC becomes critical in managing the cache data in an efficient way. Though prominent, least recently used (LRU) algorithm has some issues with applications which do not follow the temporal locality pattern. This study proposes a modification to the LRU algorithm where a random selection of the victim from 'N' LRU blocks yields better results than the conventional method. The evaluation of the algorithm is carried out using Multi2sim simulator using Parsec and Splash2 benchmarks. The results show an overall performance improvement in hit ratio up to 6% and 2% over LRU for PARSEC and SPLASH2 benchmarks respectively.

Loop Parallelism Maximization for Multimedia Data Processing in Mobile Vehicular Clouds

Mobile vehicular cloud has become popular with the rapid development of cloud computing and mobile computing. Nested loops are usually the most critical part in multimedia and high performance Digital Signal Processing (DSP) systems which are widely used in vehicular applications and systems. In order to further explore the parallelism in nested loops, we study how to maximize the system performance with considering the energy reduction for applications on Chip Multiprocessor (CMP) architectures. We propose an algorithm Energy-Aware Loop Parallelism Maximization (EALPM) to maximize the system performance with the consideration of energy reduction for applications with multidimensional nested loops. Our experiment shows that using the EALPM algorithm significantly improves both performance and energy consumption on average in comparision to other algorithms.

PR-LRU: partial random LRU technique for performance improvement of last level cache

As chip multiprocessors (CMP) have become eminent in all areas of computing, it is inevitable for the operating system to schedule processes efficiently on different cores. These multi-cores pose different challenges of which shared resource contention is the dominant one, as cores share resources like last level cache (LLC) and main memory. This can lead to poor and unpredictable performance of the threads running on the system. The cache replacement policy of LLC becomes critical in managing the cache data in an efficient way. Though prominent, least recently used (LRU) algorithm has some issues with applications which do not follow the temporal locality pattern. This study proposes a modification to the LRU algorithm where a random selection of the victim from 'N' LRU blocks yields better results than the conventional method. The evaluation of the algorithm is carried out using Multi2sim simulator using Parsec and Splash2 benchmarks. The results show an overall performance improvement in hit ratio up to 6% and 2% over LRU for PARSEC and SPLASH2 benchmarks respectively.

A Reuse-Degree Based Locality Classifier for Locality-Aware Data Replication

The last level cache (LLC) in shared configuration is widely used in the tiled chip multiprocessors (CMPs), which reduces the off-chip miss rate but incurs the long on-chip access latency. The state-of-the-art Locality-Aware Data Replication (LADR) scheme provides an effective tradeoff between capacity and latency through an in-hardware structure named locality classifier. However, the best Limited3 locality classifier (Limited3) in LADR equally preserves locality information of 3 cores for all cache lines indiscriminately that is superfluous for some lines reused by less than 3 cores but incomplete for other lines reused by more than 3 cores, which not only wastes the storage space but also limits the performance improvement. In this paper, we propose a novel concept of Reuse-Degree (RD) for each LLC line, since the line is loaded into LLC, to represent the number of cores that have reused the line. Then, we divide cache lines into Not Reused Line (NRL, RD = 0), Single Reused Line (SRL, RD = 1) and Multiple Reused Line (MRL, RD >= 2) based on their RDs and find that a significant fraction of LLC lines are NRLs or SRLs at any time. Based on this observation, we design a Reuse-Degree based Locality Classifier (RD_LC) for LADR. Specifically, RD_LC decouples the locality classifier from the LLC tag array and introduces two kinds of locality information arrays, single locality information array (SLIA) and complete locality information array (CLIA). Besides, RD_LC allocates a locality information entry only for the reused cache lines (SRLs or MRLs) instead of all cache lines, and assigns an SLIA entry to SRLs and a CLIA entry to MRLs. Our proposal avoids a waste of the storage space and also maintains enough locality information for the accuracy of data replication decisions. Experimental results show that our RD_LC for LADR saves 51% of the storage overhead than that of the baseline Limited3 locality classifier with a performance improvement and a network traffic reduction by 7.56% and 3.33 % respectively.

ReD: A reuse detector for content selection in exclusive shared last-level caches

The reference stream reaching a chip multiprocessor Shared Last-Level Cache (SLLC) shows poor temporal locality, making conventional cache management policies inefficient. Few proposals address this problem for exclusive caches. In this paper, we propose the Reuse Detector (ReD), a new content selection mechanism for exclusive hierarchies that leverages reuse locality at the SLLC, a property that states that blocks referenced more than once are more likely to be accessed in the near future. Being placed between each L2 private cache and the SLLC, ReD prevents the insertion of blocks without reuse into the SLLC. It is designed to overcome problems affecting similar recent mechanisms (low accuracy, reduced visibility window and detector thrashing). ReD improves performance over other state-of-the-art proposals (CHAR, Reuse Cache and EAF cache). Compared with the baseline system with no content selection, it reduces the SLLC miss rate (MPI) by 10.1% and increases harmonic IPC by 9.5%.

A low power high speed MTJ based non-volatile SRAM cell for energy harvesting based IoT applications

Powering billions of devices is one of the most challenging barrier in achieving the future vision of IoT. Most of the sensor nodes for IoT based systems depend on battery as their power source and therefore fail to meet the design goals of lifetime power supply, cost, reliable sensing and transmission. Energy harvesting has the potential to supplant batteries and thus prevents frequent battery replacement. However, energy autonomous systems suffer from sudden power variations due to change in external natural sources and results in loss of data. The memory system is a main component which can improve or decrease performance dramatically. The latest versions of many computing system use chip multiprocessor (CMP) with on-chip cache memory organized as array of SRAM cell. In this paper, we outline the challenges involved with the efficient power supply causing power outage in energy autonomous/self-powered systems. Also, various techniques both at circuit level and system level are discussed which ensures reliable operation of IoT device during power failure. We review the emerging non-volatile memories and explore the possibility of integrating STT-MTJ as prospective candidate for low power solution to energy harvesting based IoT applications. An ultra-low power hybrid NV-SRAM cell is designed by integrating MTJ in the conventional 6T SRAM cell. The proposed LP8T2MTJ NV-SRAM cell is then analyzed using multiple key performance parameters including read/write energies, backup/restore energies, access times and noise margins. The proposed LP8T2MTJ cell is compared to conventional 6T SRAM counterpart indicating similar read and write performance. Also, comparison with the existing MTJ based NV-SRAM cells show 51–78% reduction in backup energy and 42–70% reduction in restore energy.

A large-scale nesting ring multi-chip architecture for manycore processor systems

The optical network on chip (ONoC) paradigm has emerged as a promising solution to multi-core/many-core processor systems for offering enormous bandwidth and low power consumption. As chip multiprocessors (CMPs) scale to unprecedented numbers of cores, the performance of next-generation CMPs will be bounded by the process yield and power density of single chip. In earlier work we proposed a multi-chip ONoC architecture that scales to large numbers of CMPs and delivers high performance in terms of delay and throughout. Building on that work, in this paper we propose an optimized architecture for integrating a large number of cores into chips with a novel control strategy, including a contention resolution scheme and a resource reservation scheme. The proposed control strategy is crucial to large scale ONoCs, because the resource reservation scheme ensures efficient wavelength allocation for the traffic while the contention management scheme is effective in reducing the impact of contentions. To sustain good performance and energy efficiency of large-scale ONoC, the topology is optimized to reduce the average transmission distance with minimum increase of power consumption. We evaluate the proposed architecture within a 1000-core processor system and compare it with CMesh and several previously proposed topologies with different control strategies. The simulation results show that, our new large-scale architecture can achieve better performance on throughput and delay.

Analysis and approximation of optimal co-scheduling on cmp

In recent years, the increasing design complexity and the problems of power and heat dissipation have caused a shift in processor technology to favor Chip Multiprocessors. In Chip Multiprocessors (CMP) architecture, it is common that multiple cores share some on-chip cache. The sharing may cause cache thrashing and contention among co-running jobs. Job co-scheduling is an approach to tackling the problem by assigning jobs to cores appropriately so that the contention and consequent performance degradations are minimized. This dissertation aims to tackle two of the most prominent challenges in job co-scheduling. The first challenge is in the computational complexity for determining optimal job co-schedules. This dissertation presents one of the first systematic analyses on the complexity of job co-scheduling. Besides proving the NP completeness of job co-scheduling, it introduces a set of algorithms, based on graph theory and Integer/Linear Programming, for computing optimal co-schedules or their lower bounds in scenarios with or without job migrations. For complex cases, it empirically demonstrates the feasibility for approximating the optimal schedules effectively by proposing several heuristics-based algorithms. These discoveries facilitate the assessment of job co-schedulers by providing necessary baselines, and shed insights to the development of practical co-scheduling systems. The second challenge resides in the prediction of the performance of processes co-running on a shared cache. This dissertation explores the influence on co-run performance prediction imposed by co-runners, program inputs, and cache configurations. Through a sequence of formal analysis, we derive an analytical co-run locality model, uncovering the inherent statistical connections between the data references of programs single-runs and their co-run locality. The model offers theoretical insights on co-run locality analysis and leads to a lightweight approach for fast prediction of shared cache performance. We demonstrate the effectiveness of the model in enabling proactive job co-scheduling. Together, the two-dimensional findings open up many new opportunities for cache management on modern CMP by laying the foundation for job co-scheduling, and enhancing the understanding to data locality and cache sharing significantly.

A Monolithic 3D Hybrid Architecture for Energy-Efficient Computation

The exponentially increasing performance of chip multiprocessors (CMPs) predicted by Moore's Law is no longer due to the increasing clock rate of a single CPU core, but on account of the increase of core counts in the CMP. More transistors are integrated within the same footprint area as the technology node shrinks to deliver higher performance. However, this is accompanied by higher power dissipation that usually exceeds the coping capability of inexpensive cooling techniques. This Power Wall prevents the chip from running at full speed with all the devices powered-on. This is known as the dark silicon problem. Another major bottleneck in CMP development is the imbalance between the CPU clock rate and memory access speed. This Memory Wall keeps the CPU from fully utilizing its compute power. To address both the Power and Memory Walls, we propose a monolithic 3D hybrid architecture that consists of a multi-core CPU tier, a fine-grain dynamically reconfigurable (FDR) field-programmable gate array (FPGA) tier, and multiple resistive RAM (RRAM) tiers. The FDR tier is used as an accelerator. It uses the concept of temporal logic folding to localize on-chip communication. The RRAM tiers are connected to the CPU and FDR tiers through an efficient memory interface that takes advantage of the tremendous bandwidth available from monolithic inter-tier vias and hides the latency of large data transfers. We evaluate the architecture on two types of benchmarks: compute-intensive and memory-intensive. We show that the architecture reduces both power and energy significantly at a better performance for both types of applications. Compared to the baseline, our architecture achieves an average of 43.1× and 2.5× speedup on compute-intensive and memory-intensive benchmarks, respectively. The power and energy consumption are reduced by 5.0× and 40.5×, respectively, for compute-intensive applications, and 2.0× and 4.2×, respectively, for memory-intensive applications. This translates to 1745.3× energy-delay product (EDP) improvement for compute-intensive applications and 10.5× for memory-intensive applications.

Dynamic Lifetime Reliability Management for Chip Multiprocessors

We introduce an algorithm for dynamic lifetime reliability optimization of chip multiprocessors (CMPs). The proposed dynamic reliability management (DRM) algorithm combines thread migration and dynamic voltage and frequency scaling (DVFS) as the two primary techniques to change the CMP operation. The goal is to increase the lifetime reliability of the overall system to the desired target with minimal performance degradation. We test the proposed algorithm with a variety of benchmarks on 16 and 64 core network-on-chip (NoC) based CMP architectures. Full-system based simulations using a customized GEM5 simulator demonstrate that lifetime reliability can be improved by 100 percent for an average performance penalty of 7.7 and 8.7 percent for the two CMP architectures.

Dynamic Energy Optimization in Chip Multiprocessors Using Deep Neural Networks

We investigate the use of deep neural network (DNN) models for energy optimization under performance constraints in chip multiprocessor systems. We introduce a dynamic energy management algorithm implemented in three phases. In the first phase, training data is collected by running several selected instrumented benchmarks. A training data point represents a pair of values of cores’ workload characteristics and of optimal voltage/frequency (V/F) pairs. This phase employs Kalman filtering for workload prediction and an efficient heuristic algorithm based on dynamic voltage and frequency scaling. The second phase represents the training process of the DNN model. In the last phase, the DNN model is used to directly identify V/F pairs that can achieve lower energy consumption without performance degradation beyond the acceptable threshold set by the user. Simulation results on 16 and 64 core network-on-chip based architectures demonstrate that the proposed approach can achieve up to 55 percent energy reduction for 10 percent performance degradation constraints. In addition, the proposed DNN approach is compared against existing approaches based on reinforcement learning and Kalman filtering and found that it provides average improvements in energy-delay-product (EDP) of 6.3 and 6 percent for the 16 core architecture and of 7.4 and 5.5 percent for the 64 core architecture.

Analysing the Role of Last Level Caches in Controlling Chip Temperature

Dynamic Thermal Management (DTM) has become a major concern for the chip-designers, as it becomes a challenging task in recent power densed high performance Chip Multi-Processors (CMPs), due to integration of more on-chip components to meet ever increasing demand of processing power. The increased chip temperature incorporates severe circuit errors along with significant increment in leakage power consumption. Traditional DTM techniques apply DVFS or task migration to reduce core temperature, as cores are considered as the hottest on-chip components. Additionally, to commensurate high data demand of these high performance cores, large on-chip Last Level Caches (LLCs) are attached, which are the principal contributors to the on-chip leakage power consumption and occupy the largest on-chip area. As power consumption reduction plays the pivotal role in temperature reduction, hence, this work dynamically shrinks the cache size not only to reduce leakage power consumption, but also, to create on-chip thermal buffers for reducing average chip temperature by exploiting the heat transfer physics. Cache resizing decisions are taken based upon the generated cache hotspots and/or the access patterns, during process execution. Simulation results of the proposed thermal management method are compared with an existing DVFS based method (at cores) and a prior drowsy cache based technique to show its effectiveness.

First-Last: A Cost-Effective Adaptive Routing Solution for TSV-Based Three-Dimensional Networks-on-Chip

3D integration opens up new opportunities for future multiprocessor chips by enabling fast and highly scalable 3D Network-on-Chip (NoC) topologies. However, in an aim to reduce the cost of Through-silicon via (TSV), partially vertically connected NoCs, in which only a few vertical TSV links are available, have been gaining relevance. To reliably route packets under such conditions, we introduce a lightweight, efficient and highly resilient adaptive routing algorithm targeting partially vertically connected 3D-NoCs named First-Last. It requires a very low number of virtual channels (VCs) to achieve deadlock-freedom (2 VCs in the East and North directions and 1 VC in all other directions), and guarantees packet delivery as long as one healthy TSV connecting all layers is available anywhere in the network. An improved version of our algorithm, named Enhanced-First-Last is also introduced and shown to dramatically improve performance under low TSV availability while still using less virtual channels than state-of-the-art algorithms. A comprehensive evaluation of the cost and performance of our algorithms is performed to demonstrate their merits with respects to existing solutions.

A load-balanced congestion-aware routing algorithm based on time interval in wireless network-on-chip

Network-on-chip (NoC) has been introduced to increase the performance of chip multiprocessors (CMPs) and execute parallel programs. Although NoC is known as a modular and scalable infrastructure for interconnections, there are still some challenges with conventional NoC such as high latency and power consumption due to the communication among long-distance (LD) cores. In this regard, wireless network-on-chip (WiNoC) is a potential solution that can provide high bandwidth and low latency by means of the unique features of wireless interconnects. However, wireless routers (WRs) are prone to congestion in WiNoC due to the limited number of wireless channels on a chip and shared use of these channels by all processing elements (PEs). In this study, a load-balanced time-based congestion-aware (LTCA) routing algorithm is proposed to eliminate the congestion of WRs and distribute the traffic load on the wired and wireless networks in a balanced way. LTCA is a deadlock-free routing algorithm in which only a limited number of packets are allowed to use wireless channels. The required time for transmitting the selected packets through wireless links is measured with regard to the bandwidth of the wireless channels and traffic load. Simulation results on synthetic traffic patterns and real-world 3-tuple traffic patterns indicated a considerable improvement in latency, throughput, wired and wireless link utilization and packet loss probability.

Vertical Arbitration-Free 3-D NoCs

The vertical interlayer communication channel plays a critical role in defining the performance of a 3-D network-on-chip (NoC). In this paper, an arbitration-free design for the shared vertical channels is proposed. The proposed vertical arbitration-free 3-D NoC is compared with other 3-D NoC architectures using traditional synthetic traffic patterns and Rentian traffic emulating applications for chip multiprocessors. The results of the analysis show comparable performance in throughput, energy, and latency compared to a symmetric 3-D NoC with savings up to $\approx { 20\%}$ in area. The proposed NoC is superior in performance to a 3-D NoC utilizing vertical arbitration with a similar area footprint.

Design of a Reliable Cache System for Heterogeneous CMPs

The embedded system-on-a-chip (SoC), that integrates heterogeneous processors with variation in coherence protocol, adds complexity in maintaining coherency in the data caches. It further complicates the task of coherence verification in such systems. This work targets effective solution for coherence verification in heterogeneous chip multiprocessors (CMPs) through introduction of highly efficient verification unit. It is developed around the modeling tool of cellular automaton (CA) invented by von Neumann in 1950s. The modular and cascadable structure of CA ensures high scalability and robustness in the proposed design. A CA segment is employed to analyze the states of a data block in different private caches of a heterogeneous processor cluster and to verify inconsistencies, if any, within the cluster. The outcomes of coherence verification for clusters are analyzed by the CA resulted out of augmentation of the CA segments. On the other hand, in this work, we further propose a CA-based coherence protocol processor (PP), which caters the need for determining the state of a data block with high accuracy. The PP designed for the heterogeneous CMPs, while computing the states of za block on every transaction (read/write), can capture defects, if any, and thereby realizes a fault-tolerant PP without introduction of additional hardware logic.

A novel power model for future heterogeneous 3D chip-multiprocessors in the dark silicon age

Dark silicon has recently emerged as a new problem in VLSI technology. Maximizing performance of chip-multiprocessors (CMPs) under power and thermal constraints is very challenging in the dark silicon era. Providing next-generation analytical models for future CMPs which consider the impact of power consumption of core and uncore components such as cache hierarchy and on-chip interconnect that consume significant portion of the on-chip power consumption is largely unexplored. In this article, we propose a detailed power model which is useful for future CMP power modeling. In the proposed architecture for future CMPs, we exploit emerging technologies such as non-volatile memories (NVMs) and 3D techniques to combat dark silicon. Results extracted from the simulations are compared with those obtained from the analytical model. Comparisons show that the proposed model accurately estimates the power consumption of CMPs running both multi-threaded and multi-programed workloads.

MTB-Fetch: Multithreading Aware Hardware Prefetching for Chip Multiprocessors

To fully exploit the scaling performance in Chip Multiprocessors, applications must be divided into semi-independent processes that can run concurrently on multiple cores within a system. One major class of such applications, shared-memory, multi-threaded applications, requires programmers insert thread synchronization primitives (i.e., locks, barriers, and condition variables) in their critical sections to synchronize data access between processes. For this class of applications, scaling performance requires balanced per-thread workloads with little time spent in critical sections. In practice, however, threads often waste significant time waiting to acquire locks/barriers in their critical sections, leading to thread imbalance and poor performance scaling. Moreover, critical sections often stall data prefetchers that mitigate the effects of long critical section stalls by ensuring data is preloaded in the core caches when the critical section is complete. In this paper we examine a pure hardware technique to enable safe data prefetching beyond synchronization points in CMPs. We show that successful prefetching beyond synchronization points requires overcoming two significant challenges in existing prefetching techniques. First, we find that typical data prefetchers are designed to trigger prefetches based on current misses. This approach this works well for traditional, continuously executing, single-threaded applications. However, when a thread stalls on a synchronization point, it typically does not produce any new memory references to trigger a prefetcher. Second, even in the event that a prefetch were to be correctly directed to read beyond a synchronization point, it will likely prefetch shared data from another core before this data has been written. While this prefetch would be considered “accurate” it is highly undesirable, because such a prefetch would lead to three extra “ping-pong” movements back and forth between private caches in the producing and consuming cores, incurring more latency and energy overhead than without prefetching. We develop a new data prefetcher, Multi-Thread B-Fetch (MTB-Fetch), built as an extension to a previous single-threaded data prefetcher. MTB-Fetch addresses both issues in prefetching for shared memory multi-threaded workloads. MTB-Fetch achieves a speedup of 9.3 percent for multi-threaded applications with little additional hardware.

Energy-Efficient Monolithic Three-Dimensional On-Chip Memory Architectures

Memory bandwidth is one of the major performance bottlenecks for chip multiprocessors (CMPs), which continue to integrate an increasing number of cores with the help of Moore's Law. The growing disparity between the CPU clock rate and offchip memory access speed is known as the Memory Wall. This problem has been actively studied in the past two decades. It is addressed by placing memory closer to the processor, such as stacking the memory directly on top of a CMP, thereby significantly reducing the interconnect latency between them. However, previous 3D-stacked memory architectures use through-silicon via (TSV)based three-dimensional (3-D) integration, which bonds multiple dies with TSVs that have diameters in the 1-5 μm range. Unlike TSV-based 3-D integration, monolithic 3-D integration builds device tiers sequentially on a single substrate. Different tiers are connected using monolithic inter-tier vias (MIVs), which have a diameter (around 50 nm) that is the same as that of a local via. Main memory typically consists of DRAM, which is volatile and thus requires periodic refresh to maintain the stored data. This increases both the energy consumption and access latency. However, various nonvolatile RAMs (NVRAMs) have emerged as possible universal memory technologies, which promise low power, fast read access, high density, and nonvolatility. In this paper, we present an efficient memory interface for monolithic 3D-stacked RAM (both DRAM and NVRAMs such as resistive RAM and nanotube RAM). It takes advantage of the tremendous bandwidth made available by MIVs to implement an on-chip memory bus in order to hide the latency of large data transfers. We propose a multientry row-based write buffer to increase the buffer hit rate and reduce the number of memory core accesses. We decouple read and write accesses using extra interconnects available through MIVs to increase memory throughput. We also present an adaptive power-down policy to maintain balance between energy efficiency and performance. Simulation results show that the proposed architecture can achieve both high performance and energy efficiency, and is thus attractive for low-power/high-performance computing.

PROSA: Protocol-Driven Network on Chip Architecture

Nowadays chip multiprocessors (CMPs) tend to increase the number of cores, usually implementing a distributed shared last level cache (LLC). The network on chip (NoC) is in charge of interconnecting the cores, memory controller(s) and cache banks, largely impacting memory access latency. Packet switching (PS) is typically used in NoCs but circuit switching (CS) may complement PS achieving higher performance if the circuit is established before its need. In this paper we propose PROSA, an architecture to improve memory access latency by using CS. In PROSA, the coherence protocol steers the circuit setup logic in order to configure circuits before they are needed and only for the time they are required. PROSA uses a clustered router approach where groups of four routers are clustered and their circuit control logic is combined. Based on key design decisions, we present different PROSA versions, analyzing their impact on applications and NoC performance. PROSA is able to reduce applications’ execution time by 35 percent while it significantly reduces average network flit latency by 54 percent, leading to a reduction of miss load (and store) latency of 21 percent (in CMP systems with 64 processors). PROSA needs 8.4 percent more area, but reduces power consumption by 7 percent.