Chip Multiprocessor Systems Research Articles

Proactive Load Balancing to Reduce Unnecessary Thread Migrations on Chip Multi-Processor (CMP) Systems

Proactive Load Balancing to Reduce Unnecessary Thread Migrations on Chip Multi-Processor (CMP) Systems

A tree-recursive partitioned multicast mechanism for NoC-based deep neural network accelerator

In chip multiprocessor systems (CMPs), Network on Chip (NoC) has been widely used due to its advantages of favorable reusability, high reliability, and low power consumption. Recently, using NoC platforms to accelerate deep neural networks (DNNs) has become a new trend. This design can enable the intermediate computation results of DNNs to be transmitted within the chip, reducing the number of accesses to off-chip memory. However, a large amount of one-to-many traffic in the DNN accelerator will occupy the system bandwidth, which will significantly reduce the performance of the NoC platform dominated by one-to-one traffic. To address this issue, we propose a tree-based recursive partitioning multicast scheme (TRPM), which increases the path diversity and improves the system bandwidth. We also design a single-cycle per-hop router architecture, which effectively enhances the transmission efficiency of multicast packets. Detailed simulation results show that compared with the latest tree-based multicast algorithm for DNN accelerators, our scheme reduces the number of routed packets by 35%, the classification latency by 13.5% and the average packet latency by 14.5% on average.

Notice of Violation of IEEE Publication Principles: An Energy-Efficient Heterogeneous Memory Architecture for Future Dark Silicon Embedded Chip-Multiprocessors

Main memories play an important role in overall energy consumption of embedded systems. Using conventional memory technologies in future designs in nanoscale era causes a drastic increase in leakage power consumption and temperature-related problems. Emerging non-volatile memory (NVM) technologies offer many desirable characteristics such as near-zero leakage power, high density and non-volatility. They can significantly mitigate the issue of memory leakage power in future embedded chip-multiprocessor (eCMP) systems. However, they suffer from challenges such as limited write endurance and high write energy consumption which restrict them for adoption in modern memory systems. In this article, we present a convex optimization model to design a 3D stacked hybrid memory architecture in order to minimize the future embedded systems energy consumption in the dark silicon era. This proposed approach satisfies endurance constraint in order to design a reliable memory system. Our convex model optimizes numbers and placement of eDRAM and STT-RAM memory banks on the memory layer to exploit the advantages of both technologies in future eCMPs. Energy consumption, the main challenge in the dark silicon era, is represented as a major target in this work and it is minimized by the detailed optimization model in order to design a dark silicon aware 3D Chip-Multiprocessor. Experimental results show that in comparison with the Baseline memory design, the proposed architecture improves the energy consumption and performance of the 3D CMP on average about 61.33 and 9 percent respectively.

Pre-Silicon NBTI Delay-Aware Modeling of Network-on-Chip Router Microarchitecture

With enhancement of technology downscaling and packaging allows us to put more and more transistors within the same chip area. The Moore’s law still shows the growth in yield with increasing transistor density in deep sub-micron. This increment in transistor density encourages the computer architects to design Chip Multiprocessor (CMP) systems. In recent times, industry and researchers are focusing to add more cores per chip with increasing integration. The sharp growth in core numbers in System-on-Chip (SoC) raise intense challenges for communication among the cores. The scalability, fault tolerance and higher communication bandwidth features made Network-on-chip (NoC), a popular communication architecture than conventional bus architecture. However, the rapid shrinking of CMOS design margins in deep submicron technology has made aging mechanism such as Negative Bias Temperature Instability (NBTI), a prime concern in NoC design. In this paper, we propose a novel pre-silicon, NBTI stress aware circuit-to-system level solution for a reliable NoC router. To achieve reliability objective, we develop an NBTI aging-aware timing analysis framework based on the real workload stress. Our proposal is based on findings of most stressed path and cells due to NBTI induced effects. We develop an aging-aware cell library and used the multi Vdd technique to mitigate the NBTI induced delay effects. On the basis of volume of experimental analysis, we observed that 66.67% delay increases due to NBTI induced aging effects with real workload stress for a duration of five years. Our proposed NBTI stress mitigation algorithm and technique could able to reduce aging induced delay effects by 36%, hence enhancing the router performance under stress.

Exploring a New Adaptive Routing Based on the Dijkstra Algorithm in Optical Networks-on-Chip.

The photoelectric hybrid network has been proposed to achieve the ultrahigh bandwidth, lower delay, and less power consumption for chip multiprocessor (CMP) systems. However, a large number of optical elements used in optical networks-on-chip (ONoCs) generate high transmission loss which will influence network performance severely and increase power consumption. In this paper, the Dijkstra algorithm is adopted to realize adaptive routing with minimum transmission loss of link and reduce the output power of the link transmitter in mesh-based ONoCs. The numerical simulation results demonstrate that the transmission loss of a link in optimized power control based on the Dijkstra algorithm could be maximally reduced compared with traditional power control based on the dimensional routing algorithm. Additionally, it has a greater advantage in saving the average output power of optical transmitter compared to the adaptive power control in previous studies, while the network size expands. With the aid of simulation software OPNET, the network performance simulations in an optimized network revealed that the end-to-end (ETE) latency and throughput are not vastly reduced in regard to a traditional network. Hence, the optimized power control proposed in this paper can greatly reduce the power consumption of s network without having a big impact on network performance.

Multi-Path Propagation in On-Chip Optical Wireless Links

Optical wireless networks-on-chip (OWiNoC) are considered as a possible solution to overcome the communication bottleneck due to wired interconnects in modern chip multi-processor systems. The efficient implementation of optical wireless links requires considering many different aspects, including analysis and deep understanding of the effects on the propagation of the electromagnetic field induced by the discontinuities that can be found in a realistic scenario. This letter aims at showing the impact determined by some of these discontinuities on optical links designed in Silicon-on-Insulator technology, exploiting simple point-to-point interconnections as a first example. Measurements and simulations confirm that multi-path phenomena, triggered by the multi-layer structure housing the antennas and the propagation paths, can have a serious impact on the link budget, with fading effects that may compromise the performance of the link. However, in the presented experimental results, thanks to a careful choice of the chip layer structure, received powers higher than those which could have been measured for optical links fabricated in an infinite homogeneous medium are observed, thus resulting beneficial for the connection's power budget.

A review on shared resource contention in multicores and its mitigating techniques

Chip multiprocessor (CMP) systems have become inevitable to meet high computing demands. In such systems sharing of resources is imperative for better resource utilisation. The challenge arises when various application programs running on neighbouring cores compete for these resources concurrently and introduce contention. We aim to present in a simple, lucid and captivating manner a review of previous work on contention in multicores due to various shared resources like shared caches, main memory, memory bus bandwidth, prefetchers etc. The work investigates key ideas proposed by the research community to alleviate resource contention due to these various resources, under a single umbrella. The prime objective of the study is to throw light upon the fact that, alone a single shared component is not a dominant reason for performance degradation in CMPs, rather all elements in the memory hierarchy introduce resource contention thereby affecting performance cumulatively. The work presented would assist novice readers, researchers and academicians to further serve to propose optimal policies to address contention in designing multicore applications, considering the overall impact of these resources on the performance of multicore systems.

A review on shared resource contention in multicores and its mitigating techniques

Chip multiprocessor (CMP) systems have become inevitable to meet high computing demands. In such systems sharing of resources is imperative for better resource utilisation. The challenge arises when various application programs running on neighbouring cores compete for these resources concurrently and introduce contention. We aim to present in a simple, lucid and captivating manner a review of previous work on contention in multicores due to various shared resources like shared caches, main memory, memory bus bandwidth, prefetchers etc. The work investigates key ideas proposed by the research community to alleviate resource contention due to these various resources, under a single umbrella. The prime objective of the study is to throw light upon the fact that, alone a single shared component is not a dominant reason for performance degradation in CMPs, rather all elements in the memory hierarchy introduce resource contention thereby affecting performance cumulatively. The work presented would assist novice readers, researchers and academicians to further serve to propose optimal policies to address contention in designing multicore applications, considering the overall impact of these resources on the performance of multicore systems.

Router-integrated Cache Hierarchy Design for Highly Parallel Computing in Efficient CMP Systems

In current Chip Multi-Processor (CMP) systems, data sharing existing in cache hierarchy acts as a critical issue which costs plenty of clock cycles for maintaining data coherence. Along with the integrated core number increasing, the only shared cache serves too many processing threads to maintain sharing data efficiently. In this work, an enhanced router network is integrated within the private cache level for fast interconnecting sharing data accesses existing in different threads. All sharing data in private cache level can be classified into seven access types by experimental pattern analysis. Then, both shared accesses and thread-crossed accesses can be rapidly detected and dealt with in the proposed router network. As a result, the access latency of private cache is decreased, and a conventional coherence traffic problem is alleviated. The process in the proposed path is composed of three steps. Firstly, the target accesses can be detected by exploring in the router network. Then, the proposed replacement logic can handle those accesses for maintaining data coherence. Finally, those accesses are delivered in the proposed data deliverer. Thus, the harmful data sharing accesses are solved within the first chip layer in 3D-IC structure. The proposed system is also implemented into a cycle-precise simulation platform, and experimental results illustrate that our model can improve the Instructions Per Cycle (IPC) of on-chip execution by maximum 31.85 percent, while energy consumption can be saved by about 17.61 percent compared to the base system.

Dynamic Data Allocation and Task Scheduling on Multiprocessor Systems With NVM-Based SPM

Low-power and short-latency memory access is critical to the performance of chip multiprocessor (CMP) system devices, especially to bridge the performance gap between memory and CPU. Together with increased demand for low-energy consumption and high-speed memory, scratch-pad memory (SPM) has been widely adopted in multiprocessor systems. In this paper, we employ a hybrid SPM, composed of a static random-access memory and a nonvolatile memory (NVM), to replace the cache in CMP. However, there are several challenges related to the CMP that need to be addressed, including how to dynamically assign processors to application tasks and dynamically allocate data to memories. To solve these problems based on this architecture, we propose a novel dynamic data allocation and task scheduling algorithm, i.e., dynamic greedy data allocation and task scheduling (DGDATS). Experiments on DSP benchmarks demonstrate the effectiveness and efficiency of our proposed algorithms; namely, our proposed algorithm can generate a highly efficient dynamic data allocation and task scheduling approach to minimize the total execution cost and produce the least amount of write operations on NVMs. Our extensive simulation study demonstrates that our proposed algorithm exhibits an excellent performance compared with the heuristic allocation (HA) and adaptive genetic algorithm for data allocation (AGADA) algorithms. Based on the CMP systems with hybrid SPMs, DGDATS reduces the total execution cost by 22.18% and 51.37% compared with those of the HA and AGADA algorithms, respectively. Additionally, the average number of write operations on NVM is 19.82% lower than that of HA.

Dynamic Energy Optimization in Chip Multiprocessors Using Deep Neural Networks

We investigate the use of deep neural network (DNN) models for energy optimization under performance constraints in chip multiprocessor systems. We introduce a dynamic energy management algorithm implemented in three phases. In the first phase, training data is collected by running several selected instrumented benchmarks. A training data point represents a pair of values of cores’ workload characteristics and of optimal voltage/frequency (V/F) pairs. This phase employs Kalman filtering for workload prediction and an efficient heuristic algorithm based on dynamic voltage and frequency scaling. The second phase represents the training process of the DNN model. In the last phase, the DNN model is used to directly identify V/F pairs that can achieve lower energy consumption without performance degradation beyond the acceptable threshold set by the user. Simulation results on 16 and 64 core network-on-chip based architectures demonstrate that the proposed approach can achieve up to 55 percent energy reduction for 10 percent performance degradation constraints. In addition, the proposed DNN approach is compared against existing approaches based on reinforcement learning and Kalman filtering and found that it provides average improvements in energy-delay-product (EDP) of 6.3 and 6 percent for the 16 core architecture and of 7.4 and 5.5 percent for the 64 core architecture.

PROSA: Protocol-Driven Network on Chip Architecture

Nowadays chip multiprocessors (CMPs) tend to increase the number of cores, usually implementing a distributed shared last level cache (LLC). The network on chip (NoC) is in charge of interconnecting the cores, memory controller(s) and cache banks, largely impacting memory access latency. Packet switching (PS) is typically used in NoCs but circuit switching (CS) may complement PS achieving higher performance if the circuit is established before its need. In this paper we propose PROSA, an architecture to improve memory access latency by using CS. In PROSA, the coherence protocol steers the circuit setup logic in order to configure circuits before they are needed and only for the time they are required. PROSA uses a clustered router approach where groups of four routers are clustered and their circuit control logic is combined. Based on key design decisions, we present different PROSA versions, analyzing their impact on applications and NoC performance. PROSA is able to reduce applications’ execution time by 35 percent while it significantly reduces average network flit latency by 54 percent, leading to a reduction of miss load (and store) latency of 21 percent (in CMP systems with 64 processors). PROSA needs 8.4 percent more area, but reduces power consumption by 7 percent.

FBNoC: FPGA-based network on chip emulator for full-system architectural simulation of many-core systems

SoCs and chip multi processors (CMPs) usually employ a scalable interconnection network (Network-on-chip or NoC) as a communication medium. Existing simulators that can simulate such systems are mostly SW based and as such are very slow. Fast FPGA-based hardware simulators of CMP systems are gaining popularity because they allow fast and efficient exploration of several design points with multiple benchmarks. This entail emulating the NoC in these systems with different parameters and configurations. Existing hardware NoC simulators that are implemented on Field Programmable Gate Arrays (FPGAs) are not readily integrate-able with other FPGA-based multicore architectural simulators. In this work, a new FPGA-based NoC emulator (FBNoC) that can be integrated with multicore architectural simulators is proposed. It can also be used as a standalone NoC simulator. Utilizing a parametrizable latency model, FBNoC can deliver packets and accurately estimate their latencies for several NoC topologies, configurations, and parameters without the need for re-synthesize nor FPGA re-configuration. It also employs a novel multi-local port per NoC node strategy combined with two bidirectional ring networks to reduce the FPGA resource utilization while increasing the simulation speed. Requiring less FPGA resources than other FPGA-based NoC simulators, the proposed emulator can achieve more than 20,000x speedup over the popular SW NoC simulator Booksim.

SynchroTrace

Trace-driven simulation of chip multiprocessor (CMP) systems offers many advantages over execution-driven simulation, such as reducing simulation time and complexity, allowing portability, and scalability. However, trace-based simulation approaches have difficulty capturing and accurately replaying multithreaded traces due to the inherent nondeterminism in the execution of multithreaded programs. In this work, we present SynchroTrace, a scalable, flexible, and accurate trace-based multithreaded simulation methodology. By recording synchronization events relevant to modern threading libraries (e.g., Pthreads and OpenMP) and dependencies in the traces, independent of the host architecture, the methodology is able to accurately model the nondeterminism of multithreaded programs for different hardware platforms and threading paradigms. Through capturing high-level instruction categories, the SynchroTrace average CPI trace Replay timing model offers fast and accurate simulation of many-core in-order CMPs. We perform two case studies to validate the SynchroTrace simulation flow against the gem5 full-system simulator: (1) a constraint-based design space exploration with traditional CMP benchmarks and (2) a thread-scalability study with HPC-representative applications. The results from these case studies show that (1) our trace-based approach with trace filtering has a peak speedup of up to 18.7× over simulation in gem5 full-system with an average of 9.6× speedup, (2) SynchroTrace maintains the thread-scaling accuracy of gem5 and can efficiently scale up to 64 threads, and (3) SynchroTrace can trace in one platform and model any platform in early stages of design.

Energy Optimization for Data Allocation With Hybrid SRAM+NVM SPM

The gradually widening disparity in the speed of the CPU and memory has become a bottleneck for the development of chip multiprocessor (CMP) systems. Increasing penalties caused by frequent on-chip memory access have raised critical challenges in delivering high memory access performance with tight energy and latency budgets. To overcome the memory wall and energy wall issues, this paper adopts CMP systems with hybrid scratchpad memories (SPMs), which are configured from SRAM and nonvolatile memory. Based on this architecture, we propose two novel algorithms, i.e., energy-aware data allocation (EADA) and balancing data allocation to energy and write operations (BDAEW), to perform data allocation to different memories and task mapping to different cores, reducing energy consumption and latency. We evaluate the performance of our proposed algorithms by comparison with a parallel solution that is commonly used to solve data allocation and task scheduling problems. Experiments show the merits of the hybrid SPM architecture over the traditional pure memory system and the effectiveness of the proposed algorithms. Compared with the AGADA algorithm, the EADA and BDAEW algorithms can reduce energy consumption by 23.05% and 19.41%, respectively.

Control-theoretic adaptive cache-fair scheduling of chip multiprocessor systems

In chip multiprocessor scheduling, cache memory awareness is critical. However, most existing cache-aware multiprocessor scheduling methods are static and open-loop and thus do not work well in dynamic environments. To address this challenging problem, a control-theoretic approach is presented in this paper for adaptive cache-fair multiprocessor scheduling. The adaptive cache-fair multiprocessor scheduling uses process models to estimate the shared cache size requirements and the instruction count variations of the threads running on multiprocessor cores. Then, it feeds the estimated information back to an adaptive controller. Designed from the process models, the controller decides processor cycle re-allocation through model computation to maintain the instruction counts of the multiple threads at their desired values. The effectiveness of the adaptive cache-fair multiprocessor scheduling is demonstrated through examples.

Thread Criticality Assisted Replication and Migration for Chip Multiprocessor Caches

Non-Uniform Cache Architecture (NUCA) is a viable solution to mitigate the problem of large on-chip wire delay due to the rapid increase in the cache capacity of chip multiprocessors (CMPs). Through partitioning the last-level cache (LLC) into smaller banks connected by on-chip network, the access latency will exhibit non-uniform distribution. Various works have well explored the NUCA design, including block migration, block replication and block searching. However, all of the previous mechanisms designed for NUCA are thread-oblivious when multi-threaded applications are deployed on CMP systems. Due to the interference on shared resources, threads often demonstrate unbalanced progress wherein the lagging threads with slow progress are more critical to overall performance. In this paper, we propose a novel NUCA design called thread C riticality A ssisted R eplication and M igration (CARM). CARM exploits the runtime thread criticality information as hints to adjust the block replication and migration in NUCA. Specifically, CARM aims at boosting parallel application execution through prioritizing block replication and migration for critical threads. Full-system experimental results show that CARM reduces the execution time of a set of PARSEC workloads by 13.7 and 6.8 percent on average compared with the tradition D-NUCA and Re-NUCA respectively. Moreover, CARM also consumes much less energy compared with the evaluated schemes.

Energy-Aware Data Allocation With Hybrid Memory for Mobile Cloud Systems

Resource scheduling is one of the most important issues in mobile cloud computing due to the constraints in memory, CPU, and bandwidth. High energy consumption and low performance of memory accesses have become overwhelming obstacles for chip multiprocessor (CMP) systems used in cloud systems. In order to address the daunting “memory wall” problem, hybrid on-chip memory architecture has been widely investigated recently. Due to its advantages in size, real-time predictability, power, and software controllability, scratchpad memory (SPM) is a promising technique to replace the hardware cache and bridge the processor–memory gap for CMP systems. In this paper, we present a novel hybrid on-chip SPM that consists of a static random access memory (RAM), a magnetic RAM (MRAM), and a zero-capacitor RAM for CMP systems by fully taking advantages of the benefits of each type of memory. To reduce memory access latency, energy consumption, and the number of write operations to MRAM, we also propose a novel multidimensional dynamic programming data allocation (MDPDA) algorithm to strategically allocate data blocks to each memory. Experimental results show that the proposed MDPDA algorithm can efficiently reduce the memory access cost and extend the lifetime of MRAM.

Path-Diversity-Aware Fault-Tolerant Routing Algorithm for Network-on-Chip Systems

Network-on-Chip (NoC) is the regular and scalable design architecture for chip multiprocessor (CMP) systems. With the increasing number of cores and the scaling of network in deep submicron (DSM) technology, the NoC systems become subject to manufacturing defects and have low production yield. Due to the fault issues, the reduction in the number of available routing paths for packet delivery may cause severe traffic congestion and even to a system crash. Therefore, the fault-tolerant routing algorithm is desired to maintain the correctness of system functionality. To overcome fault problems, conventional fault-tolerant routing algorithms employ fault information and buffer occupancy information of the local regions. However, the information only provides a limited view of traffic in the network, which still results in heavy traffic congestion. To achieve fault-resilient packet delivery and traffic balancing, this work proposes a Path-Diversity-Aware Fault-Tolerant Routing ( PDA-FTR ) algorithm, which simultaneously considers path diversity information and buffer information. Compared with other fault-tolerant routing algorithms, the proposed work can improve average saturation throughput by 175 percent with only 8.9 percent average area overhead and 7.1 percent average power overhead.

CUTBUF: Buffer Management and Router Design for Traffic Mixing in VNET-Based NoCs

Router's buffer design and management strongly influence energy, area and performance of on-chip networks, hence it is crucial to encompass all of these aspects in the design process. At the same time, the NoC design cannot disregard preventing network-level and protocol-level deadlocks by devoting ad-hoc buffer resources to that purpose. In chip multiprocessor systems the coherence protocol usually requires different virtual networks (VNETs) to avoid deadlocks. Moreover, VNET utilization is highly unbalanced and there is no way to share buffers between them due to the need to isolate different traffic types. This paper proposes CUTBUF , a novel NoC router architecture to dynamically assign virtual channels (VCs) to VNETs depending on the actual VNETs load to significantly reduce the number of physical buffers in routers, thus saving area and power without decreasing NoC performance. Moreover, CUTBUF allows to reuse the same buffer for different traffic types while ensuring that the optimized NoC is deadlock-free both at network and protocol level. In this perspective, all the VCs are considered spare queues not statically assigned to a specific VNET and the coherence protocol only imposes a minimum number of queues to be implemented. Synthetic applications as well as real benchmarks have been used to validate CUTBUF , considering architectures ranging from 16 up to 48 cores. Moreover, a complete RTL router has been designed to explore area and power overheads. Results highlight how CUTBUF can reduce router buffers up to 33 percent with 2 percent of performance degradation, a 5 percent of operating frequency decrease and area and power saving up to 30.6 and 30.7 percent, respectively. Conversely, the flexibility of the proposed architecture improves by 23.8 percent the performance of the baseline NoC router when the same number of buffers is used.