3D NoC Architecture Research Articles

A Comprehensive Review of Processing-in-Memory Architectures for Deep Neural Networks

This comprehensive review explores the advancements in processing-in-memory (PIM) techniques and chiplet-based architectures for deep neural networks (DNNs). It addresses the challenges of monolithic chip architectures and highlights the benefits of chiplet-based designs in terms of scalability and flexibility. This review emphasizes dataflow-awareness, communication optimization, and thermal considerations in PIM-enabled manycore architectures. It discusses tailored dataflow requirements for different machine learning workloads and presents a heterogeneous PIM system for energy-efficient neural network training. Additionally, it explores thermally efficient dataflow-aware monolithic 3D (M3D) NoC architectures for accelerating CNN inferencing. Overall, this review provides valuable insights into the development and evaluation of chiplet and PIM architectures, emphasizing improved performance, energy efficiency, and inference accuracy in deep learning applications.

RLARA: A TSV-Aware Reinforcement Learning Assisted Fault-Tolerant Routing Algorithm for 3D Network-on-Chip

A three-dimensional Network-on-Chip (3D NoC) equips modern multicore processors with good scalability, a small area, and high performance using vertical through-silicon vias (TSV). However, the failure rate of TSV, which is higher than that of horizontal links, causes unpredictable topology variations and requires adaptive routing algorithms to select the available paths dynamically. Most works have aimed at the congestion control for TSV partially 3D NoCs to bypass the TSV reliability issue, while others have focused on the fault tolerance in TSV fully connected 3D NoCs and ignored the performance degradation. In order to adequately improve reliability and performance in TSV fully connected 3D NoC architectures, we propose a TSV-aware Reinforcement Learning Assisted Routing Algorithm (RLARA) for fault-tolerant 3D NoCs. The proposed method can take advantage of both the high throughput of fully connected TSVs and the cost-effective fault tolerance of partially connected TSVs using periodically updated TSV-aware Q table of reinforcement learning. RLARA makes the distributed routing decision with the lowest TSV utilization to avoid the overheating of the TSVs and mitigate the reliability problem. Furthermore, the K-means clustering algorithm is further adopted to compress the routing table of RLARA by exploiting the routing information similarity. To alleviate the inherent deadlock issue of adaptive routing algorithms, the link Q-value from reinforcement learning is combined with the router status based in buffer utilization to predict the congestion and enable RLARA to perform best even under a high traffic load. The experimental results of the ablation study on simulator Garnet 2.0 verify the effectiveness of our proposed RLARA under different fault models, which can perform better than the latest 3D NoC routing algorithms, with up to a 9.04% lower average delay and 8.58% higher successful delivered rate.

Multiobjective piecewise regressive elitism spotted hyena optimized mapping for 3D NoC architecture design

Multiobjective piecewise regressive elitism spotted hyena optimized mapping for 3D NoC architecture design

Pareto Technique Optimization for 3d Noc Architecture

Pareto Technique Optimization for 3d Noc Architecture

ELBA-NoC: ensemble learning-based accelerator for 2D and 3D network-on-chip architectures

Network-on-chips (NoCs) have emerged as a scalable alternative to traditional bus and point-to-point architectures, it has become highly sensitive as the number of cores increases. Simulation is one of the main tools used in NoC for analysing and testing new architectures. To achieve the best performance vs. cost trade-off, simulators have become an essential tool. Software simulators are too slow for evaluating large scale NoCs. This paper presents a framework which can be used to analyse overall performance of 2D and 3D NoC architectures which is fast and accurate. This framework is named as ensemble learning-based accelerator (ELBA-NoC) which is built using random forest regression algorithm to predict parameters of NoCs. On 2D, 3D NoC architectures, ELBA-NoC was tested and the results obtained were compared with extensively used Booksim NoC simulator. The framework showed an error rate of less than 5% and an overall speedup of up to 16 K×.

ELBA-NoC: ensemble learning-based accelerator for 2D and 3D network-on-chip architectures

Network-on-chips (NoCs) have emerged as a scalable alternative to traditional bus and point-to-point architectures, it has become highly sensitive as the number of cores increases. Simulation is one of the main tools used in NoC for analysing and testing new architectures. To achieve the best performance vs. cost trade-off, simulators have become an essential tool. Software simulators are too slow for evaluating large scale NoCs. This paper presents a framework which can be used to analyse overall performance of 2D and 3D NoC architectures which is fast and accurate. This framework is named as ensemble learning-based accelerator (ELBA-NoC) which is built using random forest regression algorithm to predict parameters of NoCs. On 2D, 3D NoC architectures, ELBA-NoC was tested and the results obtained were compared with extensively used Booksim NoC simulator. The framework showed an error rate of less than 5% and an overall speedup of up to 16 K×.

LATENCY EFFICIENT MAPPING IN 3D MESH NOC INTERCONNECTION NETWORK ARCHITECTURE

The progresses and developments in the domain of semiconductor technology is about to take us into the era where there will be thousands of cores available on a System on Chip. The onset of 3D integration technologies has unlocked the doors of novel prospects for design of on-chip networks in SoCs attaining the higher efficiency in contrast to 2D integration by aptly adjusting the increased path lengths of 2D on chip archetypes. The intelligent mapping of the applications to core on a given NoC architecture can result into the improved system’s dynamic latency. The paper presents a heuristic based on branch and bound methodology for intelligent mapping of applications to cores in 3D Mesh NoC architecture. The experimental outcomes illustrate that average latency has been immensely decreased in the optimized 3D-Mesh on chip networks when compared against optimized 2D-Mesh network of the same size.

A high-level exploration tool for three-dimensional network-on-chip

The convergence of 3D technology and NoC, the so-called 3D NoC, that freedom of interconnects in the vertical dimension can create new architectures, and has already proved higher performance than traditional 2D NoCs. In this paper, we explore the architectural design of 3D NoC and introduce a compositive model of fabrication cost, network throughput and power consumption, supporting six 3D NoC architectures, 3D mesh, hyper octagon, ciliated mesh, separate mesh, generic fat tree (SPIN) and butterfly fat tree (BFT), to explore different 3D design options of 3D NoCs. The model allows partition of IPs across different dies in 3D stack. Based on the model an estimation tool, 3D-partition, is created and validated by comparing its results with those obtained from NIRGAM. Effects of various 3D NoC architectures under different 3D IC partition strategies on cost, throughput and power consumption are explored to demonstrate the utility of the tool. It provides economic and performance reference to designing 3D ICs for volume production in the future.

A high-level exploration tool for three-dimensional network-on-chip

The convergence of 3D technology and NoC, the so-called 3D NoC, that freedom of interconnects in the vertical dimension can create new architectures, and has already proved higher performance than traditional 2D NoCs. In this paper, we explore the architectural design of 3D NoC and introduce a compositive model of fabrication cost, network throughput and power consumption, supporting six 3D NoC architectures, 3D mesh, hyper octagon, ciliated mesh, separate mesh, generic fat tree (SPIN) and butterfly fat tree (BFT), to explore different 3D design options of 3D NoCs. The model allows partition of IPs across different dies in 3D stack. Based on the model an estimation tool, 3D-partition, is created and validated by comparing its results with those obtained from NIRGAM. Effects of various 3D NoC architectures under different 3D IC partition strategies on cost, throughput and power consumption are explored to demonstrate the utility of the tool. It provides economic and performance reference to designing 3D ICs for volume production in the future.

Adaptive fault-tolerant architecture and routing algorithm for reliable many-core 3D-NoC systems

During the last few decades, Three-dimensional Network-on-Chips (3D-NoCs) have been showing their advantages against 2D-NoC architectures. This is thanks to the reduced average interconnect length and lower interconnect-power consumption inherited from Three-dimensional Integrated Circuits (3D-ICs). On the other hand, questions about their reliability is starting to arise. This issue is mainly caused by their complex nature where a single faulty transistor may cause intolerable performance degradation or even the entire system collapse. To ensure their correct functionality, 3D-NoC systems must be fault-tolerant to any short-term malfunction or permanent physical damage to ensure message delivery on time while minimizing the performance degradation as much as possible.In this paper, we present a fault-tolerant 3D-NoC architecture, called 3D-Fault-Tolerant-OASIS (3D-FTO).11This project is partially supported by Competitive research funding, Ref. P1-5, Fukushima, Japan. With the aid of a light-weight routing algorithm, 3D-FTO manages to avoid the system failure at the presence of a large number of transient, intermittent, and permanent faults. Moreover, the proposed architecture is leveraging on reconfigurable components to handle the fault occurrence in links, input-buffers, and crossbar, where the faults are more often to happen. The proposed 3D-FTO system is able to work around different kinds of faults ensuring graceful performance degradation while minimizing the additional hardware complexity and remaining power-efficient.

MSP based thermal-aware mapping approach for 3D Network-on-Chip under performance constraints

Three dimensional Network-on-chip (3D NoC) is proposed as an effective architecture to optimize system performance. However, thermal issues bring significant challenges on 3D NoC due to high power density. In this paper, we propose a 3D matrix synthesis problem (MSP) based thermal-aware mapping approach under performance constraints for 3D NoC architecture to realize temperature equilibrium and achieve better performance. Genetic algorithm is taken in the approach to obtain the optimal placements. Experimental results show that the proposed approach can achieve a temperature deviation of 45.3% on average compared with the state of art thermal optimization approaches. Moreover, our approach achieves 9.43% power saving and 14.88% delay reduction.

Analytical Fault Tolerance Assessment and Metrics for TSV-Based 3D Network-on-Chip

Reliability is one of the most challenging problems in the context of three-dimensional network-on-chip (3D NoC) systems. Reliability analysis is prominent for early stages of the manufacturing process in order to prevent costly redesigns of a target system. This article classifies the potential physical faults of a baseline TSV-based 3D NoC architecture by targeting two-dimensional (2D) NoC components and their inter-die connections. In this paper, through-silicon via (TSV) issues, thermal concerns, and single event effect (SEE) are investigated and categorized, in order to propose evaluation metrics for inspecting the resiliency of 3D NoC designs. A reliability analysis for major source of faults is reported in this article separately based on their mean time to failure (MTTF). TSV failure probability induced by inductive and capacitive coupling is also discussed. Finally, the paper provides a formal reliability analysis on the aggregated faults that affect TSV. This formal analysis is critical for estimating the resiliency of different components in order to mitigate the redundancy cost of fault-tolerant design or to examine the efficiency of any proposed fault-tolerant methods for 3D NoC architectures.

A Communication Feature-Oriented 3D NoC Architecture Design

A Communication Feature-Oriented 3D NoC Architecture Design

3D NoC with Inductive-Coupling Links for Building-Block SiPs

A wireless 3D NoC architecture is described for building-block SiPs, in which the number of hardware components (or chips) in a package can be changed after chips have been fabricated. The architecture uses inductive-coupling links that can connect more than two examined dies without wire connections. Each chip has data transceivers for the uplink and downlink in order to communicate with its neighboring chips in the package. These chips form a vertical unidirectional ring network so as to fully exploit the flexibility of the wireless approach that enables us to add, remove, and swap the chips in the ring. To avoid protocol and structural deadlocks in the ring, we use bubble flow control, which does not rely on the conventional VC-based deadlock avoidance mechanism. In addition, we propose a bidirectional communication scheme to form a bidirectional ring network by using the inductive-coupling transceivers that can dynamically change the communication modes, such as TX, RX, and Idle modes. This paper illustrates the inductive-coupling transceiver circuits, which can carry high data transfer rates of up to 8 Gbps per channel, for the wireless 3D NoC. It also illustrates an implementation of a wireless 3D NoC that has on-chip routers and transceivers implemented with a 65 nm process in order to show the feasibility of our proposal. The vertical bubble flow control and conventional VC-based approach on the uni- and bidirectional ring networks are compared with the vertical broadcast bus in terms of throughput, hardware amount, and application performance using a full system multiprocessor simulator. The results show that the proposed bidirectional communication scheme efficiently improves application performance without adding any inductive-coupling transceivers. In addition, the proposed vertical bubble flow network outperforms the conventional VC-based approach by 7.9-12.5 percent with a 33.5 percent smaller router area for building-block SiPs connecting up to eight chips.

Graceful deadlock-free fault-tolerant routing algorithm for 3D Network-on-Chip architectures

Three-Dimensional Networks-on-Chip (3D-NoC) has been presented as an auspicious solution merging the high parallelism of Network-on-Chip (NoC) interconnect paradigm with the high-performance and lower interconnect-power of 3-dimensional integration circuits. However, 3D-NoC systems are exposed to a variety of manufacturing and design factors making them vulnerable to different faults that cause corrupted message transfer or even catastrophic system failures. Therefore, a 3D-NoC system should be fault-tolerant to transient malfunctions or permanent physical damages.In this paper, we present an efficient fault-tolerant routing algorithm, called Hybrid-Look-Ahead-Fault-Tolerant (HLAFT), which takes advantage of both local and look-ahead routing to boost the performance of 3D-NoC systems while ensuring fault-tolerance. A deadlock-recovery technique associated with HLAFT, named Random-Access-Buffer (RAB), is also presented. RAB takes advantage of look-ahead routing to detect and remove deadlock with no considerably additional hardware complexity. We implemented the proposed algorithm and deadlock-recovery technique on a real 3D-NoC architecture (3D-OASIS-NoC11This project is partially supported by Competitive research funding, Ref. P1-5, Fukushima, Japan.) and prototyped it on FPGA. Evaluation results show that the proposed algorithm performs better than XYZ, even when considering high fault-rates (i.e., ≥ 20%), and outperforms our previously designed Look-Ahead-Fault-Tolerant routing (LAFT) demonstrated in latency/flit reduction that can reach 12.5% and a throughput enhancement reaching 11.8% in addition to 7.2% dynamic-power saving thanks to the Power-management module integrated with HLAFT.

Architecture and design of high-throughput, low-latency, and fault-tolerant routing algorithm for 3D-network-on-chip (3D-NoC)

Despite the higher scalability and parallelism integration offered by Network-on-Chip (NoC) over the traditional shared-bus based systems, it is still not an ideal solution for future large-scale Systems-on-Chip (SoCs), due to limitations such as high power consumption, high-cost communication, and low throughput. Recently, extending 2D-NoC to the third dimension (3D-NoC) has been proposed to deal with these problems; however, 3D-NoC systems are exposed to a variety of manufacturing and design factors making them vulnerable to different faults that cause corrupted message transfer or even catastrophic system failures. Therefore, a 3D-NoC system should be fault tolerant to transient malfunctions or permanent physical damages. In this paper, we propose a low-latency, high-throughput, and fault-tolerant routing algorithm named Look-Ahead-Fault-Tolerant (LAFT). LAFT reduces the communication latency and enhances the system performance while maintaining a reasonable hardware complexity and ensuring fault tolerance. We implemented the proposed algorithm on a real 3D-NoC architecture (3D-OASIS-NoC) and prototyped it on FPGA, then we evaluated its performance over various applications. Evaluation results show that the proposed algorithm efficiently reduces the communication latency that can reach an average of 38 % and 16 %, when compared to conventional XYZ and our early designed Look-Ahead-XYZ routing algorithms, respectively, and enhances the throughput with up to 46 % and 29 %.

HETEROGENEOUS 3D NETWORK-ON-CHIP ARCHITECTURES: AREA AND POWER AWARE DESIGN TECHNIQUES

Three-dimensional Network-on-Chip (3D NoC) architectures have gained a lot of popularity to solve the on-chip communication delays of next generation System-on-Chip (SoC) systems. However, the vertical interconnects of 3D NoC are expensive and complex to manufacture. Also, 3D router architecture consumes more power and occupies more area per chip floorplan compared to a 2D router. Hence, more efficient architectures should be designed. In this paper, we propose area efficient and low power 3D heterogeneous NoC architectures, which combines both the power and performance benefits of 2D routers and 3D NoC-bus hybrid router architectures in 3D NoC architectures. Experimental results show a negligible penalty (less than 5%) in average packet latency of the proposed heterogeneous 3D NoC architectures compared to typical homogeneous 3D NoCs, while the heterogeneity provides power and area efficiency of up to 61% and 19.7%, respectively.

Developing a power-efficient and low-cost 3D NoC using smart GALS-based vertical channels

Shorter global interconnects enable 3D NoC structures to offer higher performance, improved packaging density, and lower interconnect power consumption to CMPs and SoCs compared to their 2D counterparts. However, substantial challenges such as high peak temperatures, power densities and area footprints of vertical interconnects in each layer cannot be ignored. In this paper, a power and area efficient 3D NoC architecture based on power-aware Bidirectional Bisynchronous Vertical Channels (BBVC) is proposed as a solution to mitigate these challenges. Instead of using a pair of unidirectional channels for inter-layer communication, utilizing a dynamically self-configurable BBVC enables a system to benefit from low-latency nature of the vertical interconnects. In addition, based on the GALS implementation approach of the proposed channels, a forecasting-based dynamic frequency scaling technique for reducing the power consumption of the inter-layer communication is introduced. Simulation results show that the proposed architecture can reduce up to 47% through-silicon via (TSV) area footprint and up to 18% NoC power consumption with a slight performance degradation compared to a typical Symmetric 3D NoC.

Adaptive Data Compression on 3D Network-on-Chips

The three-dimensional Network-on-Chip (3D NoC) is an emerging research topic exploring the network architecture of 3D ICs that stack several wafers or dies. As such topics being extensively studied, it is found negative impacts of 3D NoC's vertical interconnects are raising concerns considering their footprint sizes and routability degradation. In our evaluation, we found such vertical bandwidth limitation can dramatically degrade system performance by up to 2.3×. Since such limitations come from physical design constraints, to mitigate performance degradation, we have no other choice but to reduce the amount of communication data on-chip, especially for those data moving vertically. In this paper, therefore, we carry out a study of data compression on 3D NoC architectures with a comprehensive set of scientific workloads. Firstly, we propose an adaptive data compression scheme for 3D NoCs, taking account of the vertical bandwidth limitation and data compressibility. Secondly, we evaluate our proposal on a 3D NoC platform and we observe that the compressibility based adaptive compression is very useful against incompressible data while the location-based adaptive compression is more effective with more layers for the 3D NoC. Thirdly, we find that in a bandwidth limited situation like a CMP with 3D NoCs having multiple connected layers, adaptive data compression with location-based control or with both compressibility and location based control is very promising if the number of layers grows.

3D Network-on-Chip Architectures Using Homogeneous Meshes and Heterogeneous Floorplans

We propose new 3D 2-layer and 3-layer NoC architectures that utilizehomogeneousregular mesh networks on a separate layer and one or twoheterogeneousfloorplanning layers. These architectures combine the benefits of compact heterogeneous floorplans and of regular mesh networks. To demonstrate these benefits, a design methodology that integrates floorplanning, routers assignment, and cycle-accurate NoC simulation is proposed. The implementation of the NoC on a separate layer offers an additional area that may be utilized to improve the network performance by increasing the number of virtual channels, buffers size, or mesh size. Experimental results show that increasing the number of virtual channels rather than the buffers size has a higher impact on network performance. Increasing the mesh size can significantly improve the network performance under the assumption that the clock frequency is given by the length of the physical links. In addition, the 3-layer architecture can offer significantly better network performance compared to the 2-layer architecture.