Limited Memory Bandwidth Research Articles

Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators

Graph Neural Networks (GNNs) have achieved remarkable accuracy in cognitive tasks such as predictive analytics on graph-structured data. Hence, they have become very popular in diverse real-world applications. However, GNN training with large real-world graph datasets in edge-computing scenarios is both memory- and compute-intensive. Traditional computing platforms such as CPUs and GPUs do not provide the energy efficiency and low latency required in edge intelligence applications due to their limited memory bandwidth. Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have been proposed as suitable candidates for accelerating AI applications at the edge, including GNN training. However, ReRAM-based PIM architectures suffer from low reliability due to their limited endurance, and low performance when they are used for GNN training in real-world scenarios with large graphs. In this work, we propose a learning-for-data-pruning framework, which leverages a trained Binary Graph Classifier (BGC) to reduce the size of the input data graph by pruning subgraphs early in the training process to accelerate the GNN training process on ReRAM-based architectures. The proposed light-weight BGC model reduces the amount of redundant information in input graph(s) to speed up the overall training process, improves the reliability of the ReRAM-based PIM accelerator, and reduces the overall training cost. This enables fast, energy-efficient, and reliable GNN training on ReRAM-based architectures. Our experimental results demonstrate that using this learning for data pruning framework, we can accelerate GNN training and improve the reliability of ReRAM-based PIM architectures by up to 1.6×, and reduce the overall training cost by 100× compared to state-of-the-art data pruning techniques.

A TabPFN-based intrusion detection system for the industrial internet of things

The industrial internet of things (IIoT) has undergone rapid growth in recent years, which has resulted in an increase in the number of threats targeting both IIoT devices and their connecting technologies. However, deploying tools to counter these threats involves tackling inherent limitations, such as limited processing power, memory, and network bandwidth. As a result, traditional solutions, such as the ones used for desktop computers or servers, cannot be applied directly in the IIoT, and the development of new technologies is essential to overcome this issue. One approach that has shown potential for this new paradigm is the implementation of intrusion detection system (IDS) that rely on machine learning (ML) techniques. These IDSs can be deployed in the industrial control system or even at the edge layer of the IIoT topology. However, one of their drawbacks is that, depending on the factory’s specifications, it can be quite challenging to locate sufficient traffic data to train these models. In order to address this problem, this study introduces a novel IDS based on the TabPFN model, which can operate on small datasets of IIoT traffic and protocols, as not in general much traffic is generated in this environment. To assess its efficacy, it is compared against other ML algorithms, such as random forest, XGBoost, and LightGBM, by evaluating each method with different training set sizes and varying numbers of classes to classify. Overall, TabPFN produced the most promising outcomes, with a 10–20% differentiation in each metric. The best performance was observed when working with 1000 training set samples, obtaining an F1 score of 81% for 6-class classification and 72% for 10-class classification.

SWattention: designing fast and memory-efficient attention for a new Sunway Supercomputer

In the past few years, Transformer-based large language models (LLM) have become the dominant technology in a series of applications. To scale up the sequence length of the Transformer, FlashAttention is proposed to compute exact attention with reduced memory requirements and faster execution. However, implementing the FlashAttention algorithm on the new generation Sunway Supercomputer faces many constraints such as the unique heterogeneous architecture and the limited memory bandwidth. This work proposes SWattention, a highly efficient method for computing the exact attention on the SW26010pro processor. To fully utilize the 6 core groups (CG) and 64 cores per CG on the processor, we design a two-level parallel task partition strategy. Asynchronous memory access is employed to ensure that memory access overlaps with computation. Additionally, a tiling strategy is introduced to determine optimal SRAM block sizes. Compared with the standard attention, SWattention achieves around 2.0x speedup for FP32 training and 2.5x speedup for mixed-precision training. The sequence lengths range from 1k to 8k and scale up to 16k without being out of memory. As for the end-to-end performance, SWattention achieves up to 1.26x speedup for training GPT-style models, which demonstrates that SWattention enables longer sequence length for LLM training.

Strengthening IoT Network Protocols: A Model Resilient Against Cyber Attacks

The pervasive Internet of Things (IoT) integration has revolutionized industries such as medicine, environmental care, and urban development. The synergy between IoT devices and 5G cellular networks has further accelerated this transformation, providing ultra-high data rates and ultra-low latency. This connectivity enables various applications, including remote surgery, autonomous driving, virtual reality gaming, and AI-driven smart manufacturing. However, IoT devices’ real-time and high-volume messaging nature exposes them to potential malicious attacks. The implementation of encryption in such networks is challenging due to the constraints of IoT devices, including limited memory, storage, and processing bandwidth. In a previous work [1], we proposed an ongoing key construction process, introducing a pivotal pool to enhance network security. The protocol is designed with a probability analysis to ensure the existence of a shared key between any pair of IoT devices, with the predefined probability set by the system designer. However, our earlier model faced vulnerabilities such as the “parking lot attack” and physical attacks on devices, as highlighted in the conclusion section. We present a complementary solution to address these issues, fortifying our previous protocol against cyber threats. Our approach involves the implementation of an internal Certification Authority (CA) that issues certificates for each IoT device before joining the network. Furthermore, all encryption keys are distributed by the primary IoT device using the Unix OS ‘passwd’ mechanism. If a device “disappears,” all encryption keys are promptly replaced, ensuring continuous resilience against potential security breaches. This enhanced protocol establishes a robust security framework for IoT networks, safeguarding against internal and external threats.

Graphics-processing-unit-accelerated ice flow solver for unstructured meshes using the Shallow-Shelf Approximation (FastIceFlo v1.0.1)

Abstract. Ice-sheet flow models capable of accurately projecting their future mass balance constitute tools to improve flood risk assessment and assist sea-level rise mitigation associated with enhanced ice discharge. Some processes that need to be captured, such as grounding-line migration, require high spatial resolution (under the kilometer scale). Conventional ice flow models mainly execute on central processing units (CPUs), which feature limited parallel processing capabilities and peak memory bandwidth. This may hinder model scalability and result in long run times, requiring significant computational resources. As an alternative, graphics processing units (GPUs) are ideally suited for high spatial resolution, as the calculations can be performed concurrently by thousands of threads, processing most of the computational domain simultaneously. In this study, we combine a GPU-based approach with the pseudo-transient (PT) method, an accelerated iterative and matrix-free solution strategy, and investigate its performance for finite elements and unstructured meshes with application to two-dimensional (2-D) models of real glaciers at a regional scale. For both the Jakobshavn and Pine Island glacier models, the number of nonlinear PT iterations required to converge a given number of vertices (N) scales in the order of 𝒪(N1.2) or better. We further compare the performance of the PT CUDA C implementation with a standard finite-element CPU-based implementation using the price-to-performance metric. The price of a single Tesla V100 GPU is 1.5 times that of two Intel Xeon Gold 6140 CPUs. We expect a minimum speedup of at least 1.5 times to justify the Tesla V100 GPU price to performance. Our developments result in a GPU-based implementation that achieves this goal with a speedup beyond 1.5 times. This study represents a first step toward leveraging GPU processing power, enabling more accurate polar ice discharge predictions. The insights gained will benefit efforts to diminish spatial resolution constraints at higher computing performance. The higher computing performance will allow for ensembles of ice-sheet flow simulations to be run at the continental scale and higher resolution, a previously challenging task. The advances will further enable the quantification of model sensitivity to changes in upcoming climate forcings. These findings will significantly benefit process-oriented sea-level-projection studies over the coming decades.

A novel time-domain in-memory computing unit using STT-MRAM

Advancements in technologies like Big Data, IoT, and AI have revealed a bottleneck in traditional von-Neumann architecture, resulting in high energy consumption and limited memory bandwidth. In-memory computing (IMC) presents a promising solution by enabling computations directly within the memory, enhancing energy-efficient computing. The existing time-domain (TD)-based IMC computations either require multiple cycles for computation through a successive read/write approach or contribute to the complexity of the peripheral circuit by adopting a cumulative delay approach. In this paper, we present a novel array architecture that utilizes spin transfer torque magnetic random access memory (STT-MRAM) bit-cells, mitigating source degeneration issue. By leveraging this advanced technology and employing a TD computing scheme, we have successfully implemented various arithmetic operations, alongside a comprehensive set of Boolean logic operations. Our design demonstrates improved area and energy efficiency compared to other existing TD computing schemes. Furthermore, despite the higher delay, our parameter-driven optimization approach efficiently minimizes it. To validate our proposal, we performed simulations using the 45 nm CMOS process and the Verilog-A based magnetic tunnel junction (MTJ) compact model. Through meticulous Monte-Carlo simulations, considering CMOS variations, the results demonstrate enhanced computational accuracy with increasing Tunnel Magnetoresistance (TMR) ratio, showcasing the potential of our architecture in advancing the field of computing.

Enabling memory access isolation in real-time cloud systems using Intel’s detection/regulation capabilities

The increasing interest in adopting cloud technologies for Industry 4.0 and mixed-criticality environments is paving the way for compelling new opportunities and challenges. However, cloud deployments use multicore processors that introduce interference between co-executing applications on different cores due to contention in shared resources, such as the memory subsystem. Such interference can cause critical applications to miss their deadlines.To enable the co-execution of critical and non-critical applications on the same multicore processor, we propose an approach that guarantees memory access time isolation for critical cores, while not jeopardizing the memory bandwidth of the non-critical ones. We prove the viability of our approach using Intel’s resource director technology for memory access detection and regulation. Experiments show that queue occupancy is an excellent metric to estimate the number of interfering cores co-accessing the memory. We also assess the indirect memory bandwidth limitation achievable by applying Intel’s Memory Bandwidth Allocation technology.

16-Bit (4 × 4) Optical Random Access Memory (RAM) Bank

During the past years, the limited access-times and memory bandwidth in electronic random-access memory (RAM) has triggered a concerted research effort towards deploying optical memory circuitry for exploiting the inherent high-bandwidth characteristics of light-based technologies. Yet the deployment of optical memory elements in functional optical RAM demonstrations has still been limited to single-bit capacity levels, with the few multi-bit demonstrations still offering only storage functionality without any true random-access capabilities. In this article, we demonstrate an experimental 16-bit all-optical RAM bank prototype that follows a 4 × 4 matrix organization and supports addressable random access storage of four 4-bit WDM-formatted optical Data Words at a 20 Gb/s memory-throughput. The proposed architecture constitutes of completely passive Row/Column decoding (RD/CD) subsystems, four Semiconductor Optical Amplifier Mach-Zehnder Interferometer (SOA-MZI) Access Gates (AGs) for enabling the access towards the 4 available Word Lines (WLs) and 16 all-optical Flip-Flops (AOFF) arranged in a 4 × 4 RAM bank. Each FF is incorporated in an indicative photonic integrated circuit (PIC). The experimental validation of the complete 16-bit RAM bank reveals error-free operation in Write mode with a peak power penalty between 8.3–10.2 dB at 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−9</sup> bit-error-rate (BER) measurements, indicating a roadmap for two-dimensional addressable (2D) RAM banks that can penetrate the optical computing application domain.

PISA-DMA: Processing-in-Memory Instruction Set Architecture Using DMA

Processing-in-memory (PIM) has attracted attention to overcome the memory bandwidth limitation, especially for computing memory-intensive DNN applications. Most PIM approaches use the CPU’s memory requests to deliver instructions and operands to the PIM engines, making a core busy and incurring unnecessary data transfer, thus, resulting in significant offloading overhead. DMA can resolve the issue by transferring a high volume of successive data without intervening CPU and polluting the memory hierarchy, thus perfectly fitting the PIM concept. However, the small computing resources of DRAM-based PIM devices allow us to transfer only small amounts of data at one DMA transaction and require a large number of descriptors, thus still incurring significant offloading overhead. This paper introduces PIM Instruction Set Architecture (ISA) using a DMA descriptor called PISA-DMA to express a PIM opcode and operand in a single descriptor. Our ISA makes PIM programming intuitive by thinking of committing one PIM instruction as completing one DMA transaction and representing a sequence of PIM instructions using the DMA descriptor list. Also, PISA-DMA minimizes the offloading overhead while guaranteeing compatibility with commercial platforms. Our PISA-DMA eliminates the opcode offloading overhead and achieves 1.25x, 1.31x, and 1.29x speedup over the baseline PIM at the sequence length of 128 with the BERT, RoBERTa, and GPT-2 models, respectively, in ONNX runtime with real machines. Also, we study how our proposed PISA affects performance in compiler optimization and show that the operator fusion of matrix-matrix multiplication and element-wise addition achieved 1.04x speedup, a similar performance gain using conventional ISAs.

CoMeT: An Integrated Interval Thermal Simulation Toolchain for 2D, 2.5D, and 3D Processor-Memory Systems

Processing cores and the accompanying main memory working in tandem enable modern processors. Dissipating heat produced from computation remains a significant problem for processors. Therefore, the thermal management of processors continues to be an active subject of research. Most thermal management research is performed using simulations, given the challenges in measuring temperatures in real processors. Fast yet accurate interval thermal simulation toolchains remain the research tool of choice to study thermal management in processors at the system level. However, the existing toolchains focus on the thermal management of cores in the processors, since they exhibit much higher power densities than memory. The memory bandwidth limitations associated with 2D processors lead to high-density 2.5D and 3D packaging technology: 2.5D packaging technology places cores and memory on the same package; 3D packaging technology takes it further by stacking layers of memory on the top of cores themselves. These new packagings significantly increase the power density of the processors, making them prone to overheating. Therefore, mitigating thermal issues in high-density processors (packaged with stacked memory) becomes even more pressing. However, given the lack of thermal modeling for memories in existing interval thermal simulation toolchains, they are unsuitable for studying thermal management for high-density processors. To address this issue, we present the first integrated Core and Memory interval Thermal ( CoMeT ) simulation toolchain. CoMeT comprehensively supports thermal simulation of high- and low-density processors corresponding to four different core-memory (integration) configurations—off-chip DDR memory, off-chip 3D memory, 2.5D, and 3D. CoMeT supports several novel features that facilitate overlying system research. CoMeT adds only an additional ~5% simulation-time overhead compared to an equivalent state-of-the-art core-only toolchain. The source code of CoMeT has been made open for public use under the MIT license.

A Survey of Network-Based Hardware Accelerators

Many practical data-processing algorithms fail to execute efficiently on general-purpose CPUs (Central Processing Units) due to the sequential matter of their operations and memory bandwidth limitations. To achieve desired performance levels, reconfigurable (FPGA (Field-Programmable Gate Array)-based) hardware accelerators are frequently explored that permit the processing units’ architectures to be better adapted to the specific problem/algorithm requirements. In particular, network-based data-processing algorithms are very well suited to implementation in reconfigurable hardware because several data-independent operations can easily and naturally be executed in parallel over as many processing blocks as actually required and technically possible. GPUs (Graphics Processing Units) have also demonstrated good results in this area but they tend to use significantly more power than FPGA, which could be a limiting factor in embedded applications. Moreover, GPUs employ a Single Instruction, Multiple Threads (SIMT) execution model and are therefore optimized to SIMD (Single Instruction, Multiple Data) operations, while in FPGAs fully custom datapaths can be built, eliminating much of the control overhead. This review paper aims to analyze, compare, and discuss different approaches to implementing network-based hardware accelerators in FPGA and programmable SoC (Systems-on-Chip). The performed analysis and the derived recommendations would be useful to hardware designers of future network-based hardware accelerators.

Asynchronous task based Eulerian-Lagrangian parallel solver for combustion applications

Multiphysics applications often require the use of intimately coupled solvers. The application studied here makes use of an Eulerian solver to model fluid flow and combustion and a Lagrangian solver to model spray droplets. These are then implemented within one code to solve gas turbine combustion problems. However, large scale simulations where the flow and spray are within the same computational process can be expensive as the parallel solution does not scale well due to the poor load balancing of the spray particles. This is overcome by an asynchronous task-based Eulerian-Lagrangian (ATEL) approach where separate computational processes are used so that each solver can use an appropriate technique to partition the problem. Previously, this has been shown to overcome the load balancing problem but was restricted to a single computational node where shared memory could be used to transfer data. This work expands the methodology to work on large scale HPC facilities using a combination of shared memory and high speed interconnect to transfer data. The parallel methodology exploits one-sided shared memory communication when the corresponding processes are located within a computer node, otherwise it falls back to a conventional pair of send/receive. Also an hierarchical partitioning procedure is proposed that ensures that groups of parallel subdomains with high connectivity are placed on a compute node. Results are shown for two combustor cases: the DLR generic single sector combustor with an injection process that resembles a prefilming airblast atomiser which is found in many modern civil aircraft engines and a bluff-body swirl burner with a single source of fuel injection resembling a pressure atomiser. Both single sector and three sector combustor configurations have been used to carry out the performance studies. All performance cases have been tested with three different solver configurations: a) base-line Eulerian-Lagrangian solver b) ATEL and c) the baseline Eulerian solver without spray. The unstructured grids varied from 7M cells to 84M cells. In all cases the ATEL solution with flow, combustion and spray scaled identically to when solving flow and combustion alone. In fact, due to the memory bandwidth limitations of multicore processors, reducing the number of cores allocated to the flow and combustion to allocate some cores for the spray, hardly affected the computational speed of the flow solution, and due to the overlap of the spray calculation meant that the coupled Eulerian-Lagrangian solution could be achieved at almost no cost penalty to the Eulerian on its own. The choice of how to split the cores across the two solvers was considered by proposing a simple model to estimate the cost of each solver. Timing measurements show, that for the cases considered, the overall computational time is only weakly sensitive to this choice.

DLUX: A LUT-Based Near-Bank Accelerator for Data Center Deep Learning Training Workloads

The frequent data movement between the processor and the memory has become a severe performance bottleneck for deep neural network (DNN) training workloads in data centers. To solve this off-chip memory access challenge, the 3-D stacking processing-in-memory (3D-PIM) architecture provides a viable solution. However, existing 3D-PIM designs for DNN training suffer from the limited memory bandwidth in the base logic die. To overcome this obstacle, integrating the DNN related logic near each memory bank becomes a promising yet challenging solution, since naively implementing the floating-point (FP) unit and the cache in the memory die incurs a large area overhead. To address these problems, we propose DLUX, a high performance and energy-efficient 3D-PIM accelerator for DNN training using the near-bank architecture. From the hardware perspective, to support the FP multiplier with low area overhead, an in-DRAM lookup table (LUT) mechanism is invented. Then, we propose to use a small scratchpad buffer together with a lightweight transformation engine to exploit the locality and enable flexible data layout without the expensive cache. From the software aspect, we split the mapping/scheduling tasks during DNN training into intralayer and interlayer phases. During the intralayer phase, to maximize data reuse in the LUT buffer and the scratchpad buffer, achieve high concurrency, and reduce data movement among banks, a 3D-PIM customized loop tiling technique is adopted. During the interlayer phase, efficient techniques are invented to ensure the input-output data layout consistency and realize the forward-backward layout transposition. Experiment results show that DLUX can reduce FP32 multiplier area overhead by 60% against the direct implementation. Compared with a Tesla V100 GPU, end-to-end evaluations show that DLUX can provide on average 6.3× speedup and 42× energy efficiency improvement.

Design of an energy-efficient binarized convolutional neural network accelerator using a nonvolatile field-programmable gate array with only-once-write shifting

This paper presents an energy-efficient hardware accelerator for binarized convolutional neural networks (BCNNs). In this BCNN accelerator, a data-shift operation becomes dominant to effectively control input/weight-data streams under limited memory bandwidth. A magnetic-tunnel-junction (MTJ)-based nonvolatile field-programmable gate array (NV-FPGA), where the amount of stored-data updating is minimized in a configurable logic block, is a well-suited hardware platform for implementing such a BCNN accelerator. Owing to the nonvolatile storage capability of the NV-FPGA, not only power consumption in the data-shift operation but also standby power consumption in the idle function block is reduced without losing internal data. It is demonstrated under 45 nm complementary metal–oxide–semiconductor/MTJ process technologies that the energy consumption of the proposed BCNN accelerator is 50.7% lower than that of a BCNN accelerator using a conventional static-random-access-memory-based FPGA.

Evaluation of Static Mapping for Dynamic Space-Shared Multi-task Processing on FPGAs

Whilst FPGAs have been used in cloud ecosystems, it is still extremely challenging to achieve high compute density when mapping heterogeneous multi-tasks on shared resources at runtime. This work addresses this by treating the FPGA resource as a service and employing multi-task processing at the high level, design space exploration and static off-line partitioning in order to allow more efficient mapping of heterogeneous tasks onto the FPGA. In addition, a new, comprehensive runtime functional simulator is used to evaluate the effect of various spatial and temporal constraints on both the existing and new approaches when varying system design parameters. A comprehensive suite of real high performance computing tasks was implemented on a Nallatech 385 FPGA card and show that our approach can provide on average 2.9 × and 2.3 × higher system throughput for compute and mixed intensity tasks, while 0.2 × lower for memory intensive tasks due to external memory access latency and bandwidth limitations. The work has been extended by introducing a novel scheduling scheme to enhance temporal utilization of resources when using the proposed approach. Additional results for large queues of mixed intensity tasks (compute and memory) show that the proposed partitioning and scheduling approach can provide higher than 3 × system speedup over previous schemes.

Skyrmion Logic-In-Memory Architecture for Maximum/Minimum Search

In modern computing systems there is the need to utilize a large amount of data in maintaining high efficiency. Limited memory bandwidth, coupled with the performance gap between memory and logic, impacts heavily on algorithms performance, increasing the overall time and energy required for computation. A possible approach to overcome such limitations is Logic-In-Memory (LIM). In this paper, we propose a LIM architecture based on a non-volatile skyrmion-based recetrack memory. The architecture can be used as a memory or can perform advanced logic functions on the stored data, for example searching for the maximum/minimum number. The circuit has been designed and validated using physical simulations for the memory array together with digital design tools for the control logic. The results highlight the small area of the proposed architecture and its good energy efficiency compared with a reference CMOS implementation.

Development of compression algorithms for hyperspectral aerospace images based on discrete orthogonal transformations

The paper describes the development of compression algorithms for hyperspectral aerospace images based on discrete orthogonal transformations for the purpose of subsequent compression in Earth remote sensing systems. As compression algorithms necessary to reduce the amount of transmitted information, it is proposed to use the developed compression methods based on Walsh-Hadamard transformations and discrete-cosine transformation. The paper considers a methodology for developing lossy and high-quality compression algorithms during recovery, taking into account which an adaptive algorithm for compressing hyperspectral AI and the generated quantization table has been developed. The conducted studies have shown that the proposed lossy algorithms have sufficient efficiency for use and can be applied when transmitting hyperspectral remote sensing data in conditions of limited buffer memory capacity and bandwidth of the communication channel.

Improving Network Slimming With Nonconvex Regularization

Convolutional neural networks (CNNs) have developed to become powerful models for various computer vision tasks ranging from object detection to semantic segmentation. However, most of the state-of-the-art CNNs cannot be deployed directly on edge devices such as smartphones and drones, which need low latency under limited power and memory bandwidth. One popular, straightforward approach to compressing CNNs is network slimming, which imposes $\ell _{1}$ regularization on the channel-associated scaling factors via the batch normalization layers during training. Network slimming thereby identifies insignificant channels that can be pruned for inference. In this paper, we propose replacing the $\ell _{1}$ penalty with an alternative nonconvex, sparsity-inducing penalty in order to yield a more compressed and/or accurate CNN architecture. We investigate $\ell _{p} (0 , transformed $\ell _{1}$ ( $\text{T}\ell _{1}$ ), minimax concave penalty (MCP), and smoothly clipped absolute deviation (SCAD) due to their recent successes and popularity in solving sparse optimization problems, such as compressed sensing and variable selection. We demonstrate the effectiveness of network slimming with nonconvex penalties on three neural network architectures – VGG-19, DenseNet-40, and ResNet-164 – on standard image classification datasets. Based on the numerical experiments, $\text{T}\ell _{1}$ preserves model accuracy against channel pruning, $\ell _{1/2, 3/4}$ yield better compressed models with similar accuracies after retraining as $\ell _{1}$ , and MCP and SCAD provide more accurate models after retraining with similar compression as $\ell _{1}$ . Network slimming with $\text{T}\ell _{1}$ regularization also outperforms the latest Bayesian modification of network slimming in compressing a CNN architecture in terms of memory storage while preserving its model accuracy after channel pruning.

Optimizing Data Pipeline Performance in Modern GPU Architectures

Optimizing data pipeline performance in modern GPU architectures is critical for achieving high computational throughput and efficient resource utilization in data-intensive applications. With the rise of deep learning, scientific simulations, and real-time analytics, GPUs have become integral in accelerating data processing tasks. However, ensuring optimal performance involves addressing several challenges, such as memory bandwidth limitations, data transfer bottlenecks between CPU and GPU, and efficient parallel execution of workloads. This research explores techniques for improving data pipeline performance by focusing on memory management, load balancing, and task scheduling. One key strategy is optimizing data movement through techniques like memory coalescing, which minimizes access latency, and overlapping data transfers with computation. Furthermore, leveraging the architectural advances in modern GPUs, such as unified memory and NVLink, can significantly reduce data transfer overhead. Task parallelism and efficient workload distribution across multiple GPU cores also play a crucial role in enhancing pipeline throughput. Additionally, the study highlights the importance of tuning GPU kernels and optimizing data preprocessing steps to ensure minimal latency and maximum throughput. By adopting advanced profiling tools and performance monitoring techniques, bottlenecks can be identified, and pipeline optimization strategies can be fine-tuned. The findings presented provide a comprehensive approach for designing and optimizing data pipelines, leading to significant performance improvements in GPU-based systems, ultimately driving the next generation of high-performance computing applications.

Effects of Using a Memory Stalled Core for Handling MPI Communication Overlapping in the SOR Solver on SX-ACE and SX-Aurora TSUBASA

Modern high-performance computing (HPC) systems consist of a large number of nodes featuring multi-core processors. Many computational fluid dynamics (CFD) codes utilize a Message Passing Interface (MPI) to exploit the potential of such systems. In general, the MPI communication costs increase as the number of MPI processes increases. In this paper, we discuss performance of the code in which a core is used as a dedicated communication core when the core cannot contribute to the performance improvement due to memory-bandwidth limitations. By using the dedicated communication core, the communication operations are overlapped with computation operations, thus enabling highly efficient computation by exploiting the limited memory bandwidth and idle cores. The performance evaluation shows that this code can hide the MPI communication times of 90% on the supercomputer SX-ACE system and 80% on the supercomputer SX-Aurora TSUBASA system, and the performance of the successive over-relaxation (SOR) method is improved by 32% on SX-ACE and 20% on SX-Aurora TSUBASA.