Many-core Architectures Research Articles

SongC: A Compiler for Hybrid Near-Memory and In-Memory Many-Core Architecture

SongC: A Compiler for Hybrid Near-Memory and In-Memory Many-Core Architecture

Exploring distance-based wireless transceiver placements for wireless network-on-chip architecture with deterministic routing algorithms

Network-on-chip (NoC) technology is crucial for integrating multiple embed-ded computing cores onto a single chip. Consequently, this has led to the de-velopment of the wireless network-on-chip (WiNoC) concept, seen as a promis-ing strategy to overcome scalability issues in communication systems withinchips for future many-core architectures. This research analyses the impactof wireless transceiver subnet clustering on the hundred-core mesh-structuredWiNoC architecture. The study aims to examine the effects of distance-basedwireless transceiver placements on the transmission delay, network throughput,and energy consumption within a mesh wireless NoC architecture featuring ahundred cores, under specific routing strategies: X-Y, west-first, negative-first,and north-last. This research investigates the impact of positioning radio sub-nets at the farthest, farther, nearest, and closest positions within an architectureequipped with four wireless transceivers. The Noxim simulator was utilised tosimulate the analysed wireless transceiver placements within the hundred-coremesh-structured WiNoC designs, with the objective of validating the results.The architecture with the wireless transceivers positioned at midway proxim-ity (nearer and further) demonstrated the best performance, as evidenced by thelowest latencies for all evaluated deterministic routing algorithms, according tothe simulation outcomes.

A high-level simulator for Network-on-Chip

This study presents a high-level simulator for Network-on-Chip (NoC), designed for many-core architectures, and integrated with the PlatEMO platform. The motivation for developing this tool arose from the need to evaluate the behavior of application mapping algorithms and the routing, both aspects are essential in the implementation and design of NoC architectures. This analysis underscored the importance of having effective NoC simulators as tools that allow for studying and comparing various network technologies while ensuring a controlled simulation environment. During this investigation and evaluation, some simulators, such as Noxim, NoCTweak, and NoCmap, among others, offered configurable parameters for application traffic, options to synthetically define topology and packet traffic patterns. Additionally, they include mapping options that optimize latency and energy consumption, routing algorithms, technological settings such as the CMOS process, and measurement options for evaluating performance metrics such as throughput and power usage. However, while these simulators meet detailed technical demands, they are mostly restricted to analyzing the low-level elements of the architecture, thus hindering quick and easy under- standing for non-specialists. This insight underscored the challenge in developing a tool that balances detailed analysis with a comprehensive learning perspective, considering the specific restrictions of each simulator analyzed. Experiments demonstrated the proposed simulator efficacy in handling algorithms like GA, PSO, and SA variant, and, surprisingly, revealed limitations of the XY algorithm in Mesh topologies, indicating the need for further investigation to confirm these findings. Future work will expand the simulator functionalities, incorporating a broader range of algorithms and performance metrics, to establish it as an indispensable tool for research and development in NoCs.

NxtSPR: A deadlock-free shortest path routing dedicated to relaying for Triplet-Based many-core Architecture

Deadlock-free routing is a significant challenge in Network-on-Chip (NoC) design as it affects the network’s latency, power consumption, and load balance, impacting the performance of multi-processor systems-on-chip. However, achieving deadlock-free routing will routinely result in expensive overhead as previous solutions either sacrifice performance or power efficiency to proactively avoid deadlocks or impose high hardware complexity to resolve deadlocks when they occur reactively. Utilizing the various characteristics of NoC to implement deadlock-free routing can be significantly more cost-effective with less impact on performance. This paper proposes a relay routing algorithm (NxtSPR) with a shortest path property and a deadlock prevention mechanism based on a synchronized Hamiltonian ring. The proposal is based on an in-depth study of the characteristics of a Triplet-Based many-core Architecture (TriBA) NoC. We establish various important topology-related theories and perform a formal verification (proof-based) for them. By utilizing the critical subgraph and apex of TriBA, NxtSPR can pre-calculate downstream nodes forwarding ports for packets by using a concise judgment strategy. This significantly reduces the computational overhead required for data transmission while optimizing the pipeline design of routers to decrease packet transmission latency and power consumption compared to other TriBA routing algorithms. We group the data transmissions according to the levels of maximum Hamiltonian edges a packet will traverse during its transmission life cycle. Independent data transmissions between groups can avoid mutual interference and resource competition, eliminating potential deadlocks. Gem5 simulation results show that, under the synthetic traffic patterns, compared to the representative (Table) and up-to-date (SPR4T) routing algorithms, NxtSPR achieves a 20.19%, 14.76%, and 5.54%, 4.66% reduction in average packet latency and per-packet power consumption, respectively. Moreover, it has an average of 18.50% and 4.34% improvement in throughput, as compared to them. PARSEC benchmark results show that NxtSPR reduces application runtime by up to a maximum of 22.30% and 12.82% compared to Table and SPR4T, and running the same applications with TriBA results in a maximum runtime reduction of 10.77% compared to 2D-Mesh.

A Comprehensive Review of Processing-in-Memory Architectures for Deep Neural Networks

This comprehensive review explores the advancements in processing-in-memory (PIM) techniques and chiplet-based architectures for deep neural networks (DNNs). It addresses the challenges of monolithic chip architectures and highlights the benefits of chiplet-based designs in terms of scalability and flexibility. This review emphasizes dataflow-awareness, communication optimization, and thermal considerations in PIM-enabled manycore architectures. It discusses tailored dataflow requirements for different machine learning workloads and presents a heterogeneous PIM system for energy-efficient neural network training. Additionally, it explores thermally efficient dataflow-aware monolithic 3D (M3D) NoC architectures for accelerating CNN inferencing. Overall, this review provides valuable insights into the development and evaluation of chiplet and PIM architectures, emphasizing improved performance, energy efficiency, and inference accuracy in deep learning applications.

Efficient Parallel Processing of R-Tree on GPUs

R-tree is an important multi-dimensional data structure widely employed in many applications for storing and querying spatial data. As GPUs emerge as powerful computing hardware platforms, a GPU-based parallel R-tree becomes the key to efficiently port R-tree-related applications to GPUs. However, traditional tree-based data structures can hardly be directly ported to GPUs, and it is also a great challenge to develop highly efficient parallel tree-based data structures on GPUs. The difficulty mostly lies in the design of tree-based data structures and related operations in the context of many-core architecture that can facilitate parallel processing. We summarize our contributions as follows: (i) design a GPU-friendly data structure to store spatial data; (ii) present two parallel R-tree construction algorithms and one parallel R-tree query algorithm that can take the hardware characteristics of GPUs into consideration; and (iii) port the vector map overlay system from CPU to GPU to demonstrate the feasibility of parallel R-tree. Experimental results show that our parallel R-tree on GPU is efficient and practical. Compared with the traditional CPU-based sequential vector map overlay system, our vector map overlay system based on parallel R-tree can achieve nearly 10-fold speedup.

DRLAR: A deep reinforcement learning-based adaptive routing framework for network-on-chips

Adaptive routing plays a pivotal role in the overall performance of Network-on-Chips (NoCs). However, with many-core architectures supporting complex and constantly changing traffic patterns for emerging applications, this aspect presents significant challenges. Our meticulous examination and analysis of existing heuristic adaptive routing algorithms revealed three key limiting factors: single network status metric, lack of system feedback awareness, and lack of ability to customize. Reinforcement Learning (RL) methods have demonstrated promising opportunities for exploring adaptive routing design. Deep reinforcement learning (DRL) techniques, in particular, enable efficient exploration in adaptive routing design spaces where heuristic strategies may be inadequate. This paper proposes a novel deep reinforcement learning framework for adaptive routing called DRLAR that is suitable for diversified traffic patterns and resolves multi-objective optimization simultaneously. DRLAR formulates routing as an agent, which makes routing decisions based on autonomous learning using multiple network state features and system-level feedback information. We conduct extensive experiments against state-of-the-art routing algorithms to evaluate our design. The results show that DRLAR reduces packet latency by an average of 33.3%, achieving a reduction of 41.6% and 10.5% on average under heavy synthetic traffic and the PARSEC 2.1 benchmark, respectively. We also perform a cost analysis to validate the potential implementation of DRLAR on NoCs with low computational, storage, and power.

SCIPIS: Scalable and concurrent persistent indexing and search in high-end computing systems

While it is now routine to search for data on a personal computer or discover data online, there is no such equivalent method for discovering data on large parallel and distributed file systems commonly deployed on HPC systems. In contrast to web search, which has to deal with a larger number of relatively small files, in HPC applications there is a need to also support efficient indexing of large files. We propose SCIPIS, an indexing and search framework, that can exploit the properties of modern high-end computing systems, with many-core architectures, multiple NUMA nodes and multiple NVMe storage devices. SCIPIS supports building and searching TFIDF persistent indexes, and can deliver orders of magnitude better performance than state-of-the-art approaches. We achieve scalability and performance of indexing by decomposing the indexing process into separate components that can be optimized independently, by building disk-friendly data structures in-memory that can be persisted in long sequential writes, and by avoiding communication between indexing threads that collaboratively build an index over a collection of large files. We evaluated SCIPIS with three types of datasets (logs, scientific data, and metadata), on systems with configurations up to 192-cores, 768 GiB of RAM, 8 NUMA nodes, and up to 16 NVMe drives, and achieved up to 29x better indexing while maintaining similar search latency when compared to Apache Lucene.

Large-Scale Molecular Dynamics Simulation Based on Heterogeneous Many-Core Architecture.

As the application of molecular dynamics (MD) simulations continues to evolve, the demand for accelerating large-scale simulation systems and handling of enormous simulation tasks is steadily increasing. We propose a parallel acceleration method for large-scale MD simulations based on Sunway heterogeneous many-core processors. This method integrates task scheduling, simulation calculations, and data storage, effectively tackling issues related to large-scale simulations and numerous simulation tasks. The task scheduling strategy flexibly handles tasks on various scales and enables parallel execution of multiple tasks. During the simulation calculations, we ported GROMACS to the Sunway architecture and accelerated the calculation of short-range forces through a heterogeneous processor. Our method achieves approximately 10-fold acceleration and 90% scalability when executing a single simulation task. When handling numerous simulation tasks, our method achieves parallel execution of all of the tasks with 90% scalability. By employing our method, we carried out 50 ns simulations on over 3000 distinct conotoxin structures individually within just 5 h. Additionally, we evaluated more than 200 protein-ligand complexes, and the simulation efficiency significantly exceeded that of midsized to small GPU clusters.

CINOC: Computing in Network-On-Chip With Tiled Many-Core Architectures for Large-Scale General Matrix Multiplications

CINOC: Computing in Network-On-Chip With Tiled Many-Core Architectures for Large-Scale General Matrix Multiplications

A THREE-STEP TEACHING MODEL FOR MANY-CORE CACHE MEMORY DESIGN

The shift in processor design towards manycore architectures necessitates effective management of data in cache memories, posing challenges related to cache coherence and parallel computing. This paper proposes a three-step teaching model to meet the demand for designing cache memory for many-core systems. The steps include: 1) acquiring a comprehension of an open-source simulator like FM-SIM (Flexible Multicore Cache Simulator), 2) learning the process of collecting trace files using Pin Tools, and 3) designing and integrating a new cache memory and/or cache coherence protocol into FM-SIM to assess their efficacy. The proposed model has demonstrated its capability to enable students to make significant progress in exploring and researching modern cache memory design, particularly in the context of manycore architectures.

MemPool: A Scalable Manycore Architecture With a Low-Latency Shared L1 Memory

MemPool: A Scalable Manycore Architecture With a Low-Latency Shared L1 Memory

Coupled Incomplete Cholesky and Jacobi Preconditioned Conjugate Gradient on the New Generation of Sunway Many-Core Architecture

Coupled Incomplete Cholesky and Jacobi Preconditioned Conjugate Gradient on the New Generation of Sunway Many-Core Architecture

Dynamic Power Management in Large Manycore Systems: A Learning-to-Search Framework

The complexity of manycore System-on-chips (SoCs) is growing faster than our ability to manage them to reduce the overall energy consumption. Further, as SoC design moves toward three-dimensional (3D) architectures, the core's power density increases leading to unacceptable high peak chip temperatures. In this article, we consider the optimization problem of dynamic power management (DPM) in manycore SoCs for an allowable performance penalty (say, 5%) and admissible peak chip temperature. We employ a machine learning– (ML) based DPM policy, which selects the voltage/frequency levels for different cluster of cores as a function of the application workload features such as core computation and inter-core traffic, and so on. We propose a novel learning-to-search (L2S) framework to automatically identify an optimized sequence of DPM decisions from a large combinatorial space for joint energy-thermal optimization for one or more given applications. The optimized DPM decisions are given to a supervised learning algorithm to train a DPM policy, which mimics the corresponding decision-making behavior. Our experiments on two different manycore architectures designed using wireless interconnect and monolithic 3D demonstrate that principles behind the L2S framework are applicable for more than one configuration. Moreover, L2S-based DPM policies achieve up to 30% energy-delay product savings and reduce the peak chip temperature by up to 17 °C compared to the state-of-the-art ML methods for an allowable performance overhead of only 5%.

Approaching the mapping limit with closed-loop mapping strategy for deploying neural networks on neuromorphic hardware.

The decentralized manycore architecture is broadly adopted by neuromorphic chips for its high computing parallelism and memory locality. However, the fragmented memories and decentralized execution make it hard to deploy neural network models onto neuromorphic hardware with high resource utilization and processing efficiency. There are usually two stages during the model deployment: one is the logical mapping that partitions parameters and computations into small slices and allocate each slice into a single core with limited resources; the other is the physical mapping that places each logical core to a physical location in the chip. In this work, we propose the mapping limit concept for the first time that points out the resource saving upper limit in logical and physical mapping. Furthermore, we propose a closed-loop mapping strategy with an asynchronous 4D model partition for logical mapping and a Hamilton loop algorithm (HLA) for physical mapping. We implement the mapping methods on our state-of-the-art neuromorphic chip, TianjicX. Extensive experiments demonstrate the superior performance of our mapping methods, which can not only outperform existing methods but also approach the mapping limit. We believe the mapping limit concept and the closed-loop mapping strategy can help build a general and efficient mapping framework for neuromorphic hardware.

Accelerating Graph Computations on 3D NoC-Enabled PIM Architectures

Graph application workloads are dominated by random memory accesses with the poor locality. To tackle the irregular and sparse nature of computation, ReRAM-based Processing-in-Memory (PIM) architectures have been proposed recently. Most of these ReRAM architecture designs have focused on mapping graph computations into a set of multiply-and-accumulate (MAC) operations. ReRAMs also offer a key advantage in reducing memory latency between cores and memory by allowing for PIM. However, when implemented on a ReRAM-based manycore architecture, graph applications still pose two key challenges—significant storage requirements (particularly due to wasted zero cell storage), and significant amount of on-chip traffic. To tackle these two challenges, in this article, we propose the design of a 3D NoC-enabled ReRAM-based manycore architecture. Our proposed architecture incorporates a novel crossbar-aware node reordering to reduce ReRAM storage requirements. Secondly, its 3D NoC-enabled design reduces on-chip communication latency. Our architecture outperforms the state-of-the-art in ReRAM-based graph acceleration by up to 5× in performance while consuming up to 10.3× less energy for a range of graph inputs and workloads.

SwParaFEM: a highly efficient parallel finite element solver on Sunway many-core architecture

swParaFEM: a highly efficient parallel finite element solver on Sunway many-core architecture

Domain-Specific Architectures: Research Problems and Promising Approaches

Process technology-driven performance and energy efficiency improvements have slowed down as we approach physical design limits. General-purpose manycore architectures attempt to circumvent this challenge, but they have a significant performance and energy-efficient gap compared to special-purpose solutions. Domain-specific architectures (DSAs), an instance of heterogeneous architectures, efficiently combine general-purpose cores and specialized hardware accelerators to boost energy efficiency and provide programming flexibility. Indeed, the hardware, software, and systems aspects in DSAs are highly tailored to maximize the energy efficiency of applications in a target domain. As DSAs and their conceptualization advance rapidly, there is a strong need to understand the research problems that need immediate attention. This article discusses the primary research directions in the design and runtime management of DSAs. Then, it surveys some promising approaches and highlights the outstanding research needs.

A Scalable Many-core Overlay Architecture on an HBM2-enabled Multi-Die FPGA

The overlay architecture enables to raise the abstraction level of hardware design and enhances hardware-accelerated applications’ portability. In FPGAs, there is a growing awareness of the overlay structure as typified by many-core architecture. It works in theory; however, it is difficult in practice, because it is beset with serious design issues. For example, the size of FPGAs is bigger than before. It is exacerbating the issue of the place-and-route. Besides, a single FPGA is actually the sum of small-to-middle FPGAs by advancing packaging technology like silicon interposers. Thus, the tightly coupled many-core designs will face this covert issue that the wires among the regions are extremely restricted. This article proposes efficient essential processing elements, micro-architecture design, and the interconnect architecture toward a scalable many-core overlay design. In particular, our work proposes a novel compact buffering technique to reduce memory resource utilization in tightly connected overlays while preserving computational efficiency. This technique reduces the utilization of BlockRAM to nearly 50% while achieving a best-case computational efficiency of 91.93% in a three-dimensional Jacobi benchmark. Besides, the proposed enhancements led to around 2× and 3× improvement in performance and power efficiency, respectively. Moreover, the improved scalability allowed increasing compute resources and delivering around 4× better performance and power efficiency, as compared to the baseline Dynamically Re-programmable Architecture of Gather-scatter Overlay Nodes overlay.

An Experimental Evaluation of Graph Coloring Heuristics on Multi- and Many-Core Architectures

An Experimental Evaluation of Graph Coloring Heuristics on Multi- and Many-Core Architectures