Many-core Chips Research Articles

Effectiveness of HT-assisted sinkhole and blackhole denial of service attacks targeting mesh networks-on-chip

There are ample opportunities at both design and manufacturing phases to meddle in a many-core chip system, especially its underlining communication fabric, known as the networks-on-chip (NoC), through the inclusion of malicious hardware Trojans (HT). In this paper, we focus on studying two specific HT-assisted Denial-of-Service (DoS) attacks, namely the sinkhole and blackhole attacks, that directly target the NoC of a many-core chip. As of the blackhole attacks, those intermediate routers with inserted HTs can stop forwarding data packets/flits towards the packets’ destination; instead, packets are either dropped from the network or diverted to some other malicious nodes. Sinkhole attacks, which exhibit similar attack effects as blackhole attacks, can occur when the NoC supports adaptive routing. In this case, a malicious node actively solicits packets from its neighbor nodes by pretending to have sufficient free buffer slots. Effects and efficiencies of both sinkhole and blackhole DoS attacks are modeled and quantified in this paper, and a few factors that influence attack effects are found to be critical. Through fine-tuning of these parameters, both attacks are shown to cause more damages to the NoC, measured as over 30% increase in packet loss rate. Even with current detection and defense methods in place, the packet loss rate is still remarkably high, suggesting the need of new and more effective detection and defense methods against the enhanced blackhole and sinkhole attacks as described in the paper.

Analyzing power-thermal-performance trade-offs in a high-performance 3D NoC architecture

The emergence of three-dimensional (3D) network-on-chip (NoC) has revolutionized the design of high-performance and energy efficient manycore chips. However, in general, 3D NoC architectures still suffer from high power density and the resultant thermal hotspots leading to functionality and reliability concerns over time. The power consumption and thermal profiles of 3D NoCs can be improved by incorporating a Voltage Frequency Island (VFI)-based power management strategy and Reciprocal Design Symmetry (RDS)-based floor planning. In this paper, we undertake a detailed design space exploration for 3D NoC by considering power-thermal-performance (PTP) trade-offs. We specifically consider a small-world network-enabled 3D NoC (3D SWNoC) in this performance evaluation due to its superior performance and energy-efficiency compared to any other existing 3D NoC architectures. We demonstrate that the VFI-enabled 3D SWNoC lowers the energy-delay-product (EDP) by 57.3% on an average compared to a 2D MESH without VFI. Moreover, by incorporating VFI, we reduce the maximum temperature of 3D SWNoC by 15.2% on an average compared to the non-VFI counterpart. By complementing the VFI-based power management with RDS-based floor planning, the 3D SWNoC reduces the maximum temperature by 25.1% on an average compared to the non-VFI counterpart.

Bubble Budgeting: Throughput Optimization for Dynamic Workloads by Exploiting Dark Cores in Many Core Systems

All the cores of a many-core chip cannot be active at the same time, due to reasons like low CPU utilization in server systems and limited power budget in dark silicon era. These free cores (referred to as bubbles) can be placed near active cores for heat dissipation so that the active cores can run at a higher frequency level, boosting the performance of applications that run on active cores. Budgeting inactive cores (bubbles) to applications to boost performance has the following three challenges. First, the number of bubbles varies due to open workloads. Second, communication distance increases when a bubble is inserted between two communicating tasks (a task is a thread or process of a parallel application), leading to performance degradation. Third, budgeting too many bubbles as coolers to running applications leads to insufficient cores for future applications. In order to address these challenges, in this paper, a bubble budgeting scheme is proposed to budget free cores to each application so as to optimize the throughput of the whole system. Throughput of the system depends on the execution time of each application and the waiting time incurred for newly arrived applications. Essentially, the proposed algorithm determines the number and locations of bubbles to optimize the performance and waiting time of each application, followed by tasks of each application being mapped to a core region. A Rollout algorithm is used to budget power to the cores as the last step. Experiments show that our approach achieves 50 percent higher throughput when compared to state-of-the-art thermal-aware runtime task mapping approaches. The runtime overhead of the proposed algorithm is in the order of 1M cycles, making it an efficient runtime task management method for large-scale many-core systems.

Enabling Massive Multi-Threading with Fast Hashing

The next generation of high-performance computers is expected to execute threads in orders of magnitude higher than today’s systems. Improper management of such huge amount of threads can create resource contention, leading to overall degraded system performance. By leveraging more practical approaches to distribute threads on the available resources, execution models and manycore chips are expected to overcome limitations of current systems. Here, we present DELTA—a Data-Enabled muLti-Threaded Architecture, where a producer-consumer scheme is used to execute threads via complete distributed thread management mechanism. We consider a manycore tiled-chip architecture where Network-on-Chip (NoC) routers are extended to support our execution model. The proposed extension is analysed, while simulation results confirm that DELTA can manage a large number of simultaneous threads, relying on a simple hardware structure.

HYDRA: Heterodyne Crosstalk Mitigation With Double Microring Resonators and Data Encoding for Photonic NoCs

Silicon-photonic networks on chip (PNoCs) provide high bandwidth with lower data-dependent power dissipation than does the traditional electrical NoCs (ENoCs); therefore, they are promising candidates to replace ENoCs in future manycore chips. PNoCs typically employ photonic waveguides with dense wavelength division multiplexing (DWDM) for signal traversal and microring resonators (MRs) for signal modulation. Unfortunately, DWDM increases susceptibility to intermodulation (IM) and off-resonance filtering effects, which reduce optical signal-to-noise ratio (OSNR) for photonic data transfers. Additionally, process variations (PVs) induce variations in the width and thickness of MRs causing resonance wavelength shifts, which further reduce OSNR, and create communication errors. This paper proposes a novel cross-layer framework called HYDRA to mitigate heterodyne crosstalk due to PVs, off-resonance filtering, and IM effects in PNoCs. The framework consists of two device-level mechanisms and a circuit-level mechanism to improve heterodyne crosstalk resilience in PNoCs. Simulation results on three PNoC architectures indicate that HYDRA can improve the worst case OSNR by up to 5.3× and significantly enhance the reliability of DWDM-based PNoC architectures.

Reducing bypass‐based network‐on‐chip latency using priority mechanism

In the movement from a multi-core to a many-core era, cores count on the chip increases quickly thus interconnect plays a large role in achieving the desired performance. Network-on-chip (NoC) is the most widely used interconnect as a scalable alternative for traditional shared bus in many-core chips. As the dimensions of mesh-based NoC increase, routers and links serve as a major part to achieve the desired performance and low-latency communication between cores. In this study, the authors propose an arbitration mechanism for NoC that leads to a reduction in congestion delay in routers as well as the network latency. The proposed mechanism is compatible with the bypass and baseline pipeline in routers. System simulations with Noxim demonstrate reduction in latencies and power consumption using different routing algorithms for 4×4,8×8 and 16×16 mesh topologies, as compared with a baseline router.

A deadlock-free routing algorithm for irregular 3D network-on-chips with wireless links

In recent years, the idea of wireless three-dimensional network-on-chips (3D NoCs) was promoted in order to design many-core chips with greater performance and lower energy consumption. This technology is the combination of different dies that are stacked on each other. Therefore, it is necessary to propose a suitable routing mechanism for irregular wireless 3D NoCs that can support the agnostic topologies. In this paper, we propose a deadlock-free routing algorithm for wireless 3D NoCs, called Floyd-base Inter-chip Traffic distribution (FIT), which is based on Floyd routing algorithm. In FIT algorithm, the number of hops is reduced compared to the already established deterministic algorithms; moreover, the traffic distribution is improved. Evaluation results show that our proposed routing algorithm significantly improves the performance and throughput by reducing the energy consumption, the average hop count and the communication latency.

Imitation Learning for Dynamic VFI Control in Large-Scale Manycore Systems

Manycore chips are widely employed in high-performance computing and large-scale data analysis. However, the design of high-performance manycore chips is dominated by power and thermal constraints. In this respect, voltage–frequency island (VFI) is a promising design paradigm to create scalable energy-efficient platforms. By dynamically tailoring the voltage and frequency of each island, we can further improve the energy savings within given performance constraints. Inspired by the recent success of imitation learning (IL) in many application domains and its significant advantages over reinforcement learning (RL), we propose the first architecture-independent IL-based methodology for dynamic VFI (DVFI) control in manycore systems. Due to its popularity in the EDA community, we consider an RL-based DVFI control methodology as a strong baseline. Our experimental results demonstrate that IL is able to obtain higher quality policies than RL (on average, 5% less energy with the same level of performance) with significantly less computation time and hardware area overheads (3.1X and 8.8X, respectively).

VFI-Based Power Management to Enhance the Lifetime of High-Performance 3D NoCs

The emergence of 3D network-on-chip (NoC) has revolutionized the design of high-performance and energy-efficient manycore chips. However, the anticipated performance gain can be compromised due to the degradation and failure of vertical links (VLs). The Through-Silicon-Via (TSV)-enabled VLs may fail due to workload-induced stress; the failure of a VL can affect the neighboring VLs, thereby causing a cascade of failures and reducing the lifetime of the chip. To enhance the reliability of 3D NoC-enabled manycore chips, we propose to incorporate a voltage-frequency island (VFI)-based power management strategy that helps to reduce the energy consumption and hence, the workload-induced stress of the highly utilized VLs. The adopted power-management strategy relies on control decisions about the voltage/frequency (V/F) levels on VLs. We demonstrate that compared to the well-known spare TSV allocation and adaptive routing strategies, power management is more effective in enhancing the reliability of a 3D NoC. VFI-based power management improves the reliability of the 3D NoC by one order of magnitude compared to both adaptive routing and spare allocation while running popular SPLASH-2 and PARSEC benchmarks. The principal benefit of power management is that it is capable of reducing the operating temperature of the system, which in turn enhances the Mean-Time-To-Failure (MTTF) of the VLs and reliability of the overall 3D NoC.

FoToNoC: A Folded Torus-Like Network-on-Chip Based Many-Core Systems-on-Chip in the Dark Silicon Era

Dark silicon refers to the phenomenon that a fraction of a many-core chip has to become “dark” or “dim” in order to guarantee the system to be kept in a safe temperature range and allowable power budget. Techniques have been developed to selectively activate non-adjacent cores on many-core chip to avoid temperature hotspot, while resulting unexpected increase of communication overhead due to the longer average distance between active cores, and in turn affecting application performance and energy efficiency, when Network-on-Chip (NoC) is used as a scalable communication subsystem. To address the brand-new challenges brought by dark silicon, in this paper, we present FoToNoC, a Folded Torus-like NoC, coupled with a hierarchical management strategy for heterogeneous many-core systems. On top of it, objectives of maximizing application performance, energy efficiency and chip reliability are isolated and well achieved by hardware-software co-design in several different phases, including application mapping and scheduling, cluster management and DVFS control. Evaluations on PARSEC benchmark applications demonstrate the significance of the entire strategy. Compared with state-of-the-art approaches, the proposed FoToNoC organization can achieve on average 35.4 and 35.2 percent on communication efficiency and application performance improvement, respectively, when maintaining the safe chip temperature. The hierarchical cluster-based management strategy can further reduce an average 34.6 percent of the total energy consumption with a notable reduction on the chip peak temperature. The significant achievements on system energy efficiency and the reduction on chip temperature of H.264 decoder and DSP-stone benchmarks additionally verify the effectiveness of the proposed methods.

PoweRock: Power Modeling and Flexible Dynamic Power Management for Many-Core Architectures

Energy-efficient design for green high-performance computing (HPC) is a formidable mission for today's computer scientists. Dynamic power management (DPM) is a key enabling technology involved. While DPM can have various goals for different application scenarios (e.g., enforcing an upper bound on power consumption or optimizing energy usage under a performance-loss constraint), existing DPM solutions are generally designed to meet only one goal and not adaptable to changes in optimization objectives. This paper proposes a novel flexible DPM approach based on a profile-guided dynamic voltage/frequency scaling (DVFS) scheme to meet the different goals. Our contributions include 1) an accurate power prediction model for many-core architectures; 2) a new profiling method for (distributed) shared-memory parallel applications to flexibly determine the optimal frequency and voltage for different phases of the execution; and 3) a hierarchical domain-aware power control design boosting the DPM system scalability for many-core chips. We implement the approach into a working library dubbed PoweRock and evaluate it on the Intel SCC port of the Barrelfish operating system. Experimental results obtained from several well-known benchmarks show that PoweRock attains significant energy and energy-delay product (EDP) improvements (average: 37.1% and 25.5%; best case: 64.0% and 65.4%, respectively) over a static power scheme.

Design-Space Exploration and Optimization of an Energy-Efficient and Reliable 3-D Small-World Network-on-Chip

A 3-D network-on-chip (NoC) enables the design of high performance and low power many-core chips. Existing 3-D NoCs are inadequate for meeting the ever-increasing performance requirements of many-core processors since they are simple extensions of regular 2-D architectures and they do not fully exploit the advantages provided by 3-D integration. Moreover, the anticipated performance gain of a 3-D NoC-enabled many-core chip may be compromised due to the potential failures of through-silicon-vias that are predominantly used as vertical interconnects in a 3-D IC. To address these problems, we propose a machine-learning-inspired predictive design methodology for energy-efficient and reliable many-core architectures enabled by 3-D integration. We demonstrate that a small-world network-based 3-D NoC (3-D SWNoC) performs significantly better than its 3-D MESH-based counterparts. On average, the 3-D SWNoC shows 35% energy-delay-product improvement over 3-D MESH for the PARSEC and SPLASH2 benchmarks considered in this paper. To improve the reliability of 3-D NoC, we propose a computationally efficient spare-vertical link (sVL) allocation algorithm based on a state-space search formulation. Our results show that the proposed sVL allocation algorithm can significantly improve the reliability as well as the lifetime of 3-D SWNoC.

Evaluating the Combined Effect of Memory Capacity and Concurrency for Many-Core Chip Design

Modern memory systems are structured under hierarchy and concurrency. The combined impact of hierarchy and concurrency, however, is application dependent and difficult to describe. In this article, we introduce C 2 -Bound, a data-driven analytical model that serves the purpose of optimizing many-core design. C 2 -Bound considers both memory capacity and data access concurrency. It utilizes the combined power of the newly proposed latency model, concurrent average memory access time, and the well-known memory-bounded speedup model (Sun-Ni’s law) to facilitate computing tasks. Compared to traditional chip designs that lack the notion of memory capacity and concurrency, the C 2 -Bound model finds that memory bound factors significantly impact the optimal number of cores as well as their optimal silicon area allocations, especially for data-intensive applications with a non-parallelizable sequential portion. Therefore, our model is valuable to the design of next-generation many-core architectures that target big data processing, where working sets are usually larger than the conventional scientific computing. These findings are evidenced by our detailed simulations, which show, with C 2 -Bound, the design space of chip design can be narrowed down significantly up to four orders of magnitude. C 2 -Bound analytic results can be either used in reconfigurable hardware environments or, by software designers, applied to scheduling, partitioning, and allocating resources among diverse applications.

Customizing Clos Network-on-Chip for Neural Networks

Large-scale neural network accelerators are often implemented as a many-core chip and rely on a network-on-chip to manage the huge amount of inter-neuron traffic. The baseline and different variations of the well-known mesh and tree topologies are the most popular topologies in prior many-core implementations of neural networks. However, the grid-like mesh and hierarchical tree topologies suffer from high diameter and low bisection bandwidth, respectively. In this paper, we present ClosNN, a customized Clos topology for Neural Networks. The inherent capability of Clos to support multicast and broadcast traffic in a simple and efficient way, as well as its adaptable bisection bandwidth, is the major motivation behind proposing a customized version of this topology as the communication infrastructure of large-scale neural network implementations. We compare ClosNN with some state-of-the-art NoC topologies adopted in recent neural network hardware accelerators and show that it offers lower average message hop count and higher throughput, which directly translates to faster neural information processing.

Mapping multiple applications onto 3D NoC-based MPSoCs supporting wireless links

Three-dimensional integrated circuits (3D ICs) are suitable alternatives to traditional two-dimensional (2D) ICs by leveraging its advantage of better performance and packaging; therefore, they have been highly considered by researchers. On the other hand, emerging network-on-chip (NoC) based many-core chips provides great potential for running multiple applications simultaneously. However, using this approach leads to the increase of the interference between applications, resulting in lowering the performance of each application. Hence, mapping tasks belonging to various applications onto the nodes of an architecture is a very important issue. In this study, based on partitioning concept, a novel methodology for mapping of multiple applications at run-time onto an irregular wireless 3D NoC-based multiprocessor system-on-chip (MPSoC) platform in which more than one task can be supported by each processing element (PE) was presented. In the second algorithm (enhanced irregular-partitioning best neighbor), according to the number of applications running simultaneously, the partitioning of network will be dynamically changed to minimize the communication overhead and congestion on the NoC that leads to more efficient task mapping. The simulation results reveal that the second proposed algorithm (enhanced IPBN) in comparison with NPBN (non-partitioning best neighbor) algorithm and our first proposed algorithm (basic IPBN) enhances the performance by decreasing the total execution time, average hop count, average channel load and energy consumption.

Energy-optimal configuration selection for manycore chips with variation

Operating chips at high energy efficiency is one of the major challenges for modern large-scale supercomputers. Low-voltage operation of transistors increases the energy efficiency but leads to frequency and power variation across cores on the same chip. Finding energy-optimal configurations for such chips is a hard problem. In this work, we study how integer linear programming techniques can be used to obtain energy-efficient configurations of chips that have heterogeneous cores. Our proposed methodologies give optimal configurations as compared with competent but sub-optimal heuristics while having negligible timing overhead. The proposed ParSearch method gives up to 13.2% and 7% savings in energy while causing only 2% increase in execution time of two HPC applications: miniMD and Jacobi, respectively. Our results show that integer linear programming can be a very powerful online method to obtain energy-optimal configurations.

Providing Balanced Mapping for Multiple Applications in Many-Core Chip Multiprocessors

This paper addresses the problem of balancing the on-chip packet latencies in a chip multi-processor (CMP), which is simultaneously executing multiple applications. Specifically, this paper presents a balanced application-to-core mapping algorithm that aims to minimize the maximum on-chip packet latency of all running applications. The paper starts by formulating the balanced mapping problem for CMPs and proving its NP-completeness. Next it presents an efficient heuristic algorithm for solving the aforesaid problem, which utilizes the characteristics of on-chip cache and memory accesses in CMPs and takes into account the workload variations among applications. Simulation results on PARSEC benchmark suite show that the proposed algorithm lowers the maximum average packet latency of all applications by 11 percent while cutting the standard deviation of on-chip packet latencies by 99 percent. This is achieved by very little overhead in terms of the overall packet latency and power consumption averaged over all packets.

Performance/energy aware task migration algorithm for many‐core chips

This study proposes an efficient task migration algorithm for mesh-based multi- and many-core chips. The proposed algorithm collects tasks running on a rectangular-based set of cores, that is, source sub-mesh and moves the tasks to another rectangular-based set to remove chip temperature hotspots and to provide balanced load on the chip. The proposed migration algorithm uses the concept of gathering/scattering to minimise the traffic induced by the migration. In this regard, the proposed algorithm uses a selected node in each row of the source sub-mesh to gather tasks of all cores in the same row. Selection of the gathering node is done based on its location in the row and traffic rate of other cores in the row. When gathering nodes are migrated, in the destination sub-mesh, then, they scatter their tasks according to the same pattern among the cores in their rows. Simulations of the proposed migration algorithm are done by Access Noxim simulator in a various range of network conditions with application graphs of D263DECMP3DEC, DMPEG4, and DVOPD. Results obtained from simulations show that the proposed algorithm offers 36% better performance, 28% lower energy consumption, and 7% lower temperature in comparison with the previously proposed migration algorithms.

A pareto-optimal runtime power budgeting scheme for many-core systems

Due to the ever-escalating power consumption, a significant proportion of the future many-core chips is mandatory to be switched off to meet the power budgets. This trend has brought up a paradigm shift from conventional low-power to power budgeting designs, where performance optimization needs to be performed under a tight power budget constraint. There are two key issues to be considered when moving this new design paradigm forward. Firstly, with per-core frequency scaling, the number of frequency combinations of the cores grows exponentially. As more cores are integrated onto a chip, it becomes more challenging to achieve the optimal performance over a given power budget. Secondly, the power budgets of many-core system might undergo a rapid fluctuation. Consequently, the power budgeting scheme needs to be prompt to make appropriate changes to track such power budget variation. This paper is aiming at resolving the problem of optimizing overall performance over a power budget using frequency scaling technique. To solve the problem efficiently at runtime, we propose a parallel dynamic programming network, in which the Pareto-optimal solutions can be obtained using linear time complexity. Experimental results have confirmed that the proposed approach can reduce the execution time by 45% when compared to other existing methods. The runtime overhead and hardware cost of the proposed approach are reasonably small, such as the average area and power consumption are less than 1% of the whole network-on-chip. This paper demonstrates an effective formulation for delivering Pareto-optimal solutions for power budgeting in future many-core systems.

WiSync

In shared-memory multiprocessing, fine-grain synchronization is challenging because it requires frequent communication. As technology scaling delivers larger manycore chips, such pattern is expected to remain costly to support. In this paper, we propose to address this challenge by using on-chip wireless communication. Each core has a transceiver and an antenna to communicate with all the other cores. This environment supports very low latency global communication. Our architecture, called WiSync, uses a per-core Broadcast Memory (BM). When a core writes to its BM, all the other 100+ BMs get updated in less than 10 processor cycles. We also use a second wireless channel with cheaper transfers to execute barriers efficiently. WiSync supports multiprogramming, virtual memory, and context switching. Our evaluation with simulations of 128-threaded kernels and 64-threaded applications shows that WiSync speeds-up synchronization substantially. Compared to using advanced conventional synchronization, WiSync attains an average speedup of nearly one order of magnitude for the kernels, and 1.12 for PARSEC and SPLASH-2.