Host Memory Research Articles

BRGraph: An efficient graph neural network training system by reusing batch data on GPU

SummaryWith the increasing adoption of graph neural networks (GNNs) in the community, various GPU‐based graph programming systems have been developed to improve the productivity of GNNs. However, sampling‐based GNN training is still inefficient, and we observe that the main bottleneck comes from the data transferring, where vertex features are transferred from host memory to GPU through limited bandwidth. In this article, we propose BRGraph, a sampling‐based GNN training system that supports efficient data transferring. BRGraph leverages the duplicate vertices between mini‐batches by batch reusing (BR) strategy to avoid duplicate data transmission. Furthermore, to reduce the overhead of detecting duplicate vertices, we design an efficient parallel batch reusing algorithm based on GPU. BRGraph also exploits the data reusing potential of the non‐duplicate vertex features by the two‐level batch reusing (two‐level BR) strategy. Comprehensive evaluations on three representative GNN models show that BRGraph reduces data transferring time by up to 60% and delivers up to 1.79 GNN training speedup over the state‐of‐the‐art baselines. Besides, it can save GPU memory by up to 40% while reaching the same training time compared with the static cache strategy. When applying the two‐level BR, BRGraph further reduces 20% of the data transferring time compared with the BR.

BurstZ+: Eliminating The Communication Bottleneck of Scientific Computing Accelerators via Accelerated Compression

We present BurstZ+, an accelerator platform that eliminates the communication bottleneck between PCIe-attached scientific computing accelerators and their host servers, via hardware-optimized compression. While accelerators such as GPUs and FPGAs provide enormous computing capabilities, their effectiveness quickly deteriorates once data is larger than its on-board memory capacity, and performance becomes limited by the communication bandwidth of moving data between the host memory and accelerator. Compression has not been very useful in solving this issue due to performance and efficiency issues of compressing floating point numbers, which scientific data often consists of. BurstZ+ is an FPGA-based prototype accelerator platform which addresses the bandwidth issue via a class of novel hardware-optimized floating point compression algorithm called ZFP-V. We demonstrate that BurstZ+ can completely remove the host-side communication bottleneck for accelerators, using multiple stencil kernels with a wide range of operational intensities. Evaluated against hand-optimized implementations of kernel accelerators of the same architecture, our single-pipeline BurstZ+ prototype outperforms an accelerator without compression by almost 4×, and even an accelerator with enough memory for the entire dataset by over 2×. Furthermore, the projected performance of BurstZ+ on a future, faster FPGA scales to almost 7× that of the same accelerator without compression, whose performance is still limited by the PCIe bandwidth.

Demand MemCpy: Overlapping of Computation and Data Transfer for Heterogeneous Computing

Heterogeneous computing relies on collaboration among different types of processors on shared data. In systems with discrete accelerators (e.g., GP-GPU), data sharing requires transferring a large amount of data between CPU and accelerator memories and can significantly increase the end-to-end execution time. This paper proposes a novel mechanism called <i>Demand MemCpy</i> (<i>DMC</i>) to hide the data sharing overheads. DMC copies data from host memory to accelerator memory based on demands at page granularity. It utilizes a hardware-only mechanism to fetch the requested page with a short latency and the background pre-copy to fetch related pages in advance. Our evaluation shows that DMC can reduce the end-to-end execution time of GP-GPU application by 25.4% on average by overlapping computation with data transfer and not transferring unused pages.

VEDAS: an efficient GPU alternative for store and query of large RDF data sets

Resource Description Framework (RDF) is commonly used as a standard for data interchange on the web. The collection of RDF data sets can form a large graph which consumes time to query. It is known that modern Graphic Processing Units (GPUs) can be employed to execute parallel programs in order to speedup the running time. In this paper, we propose a novel RDF data representation along with the query processing algorithm that is suitable for GPU processing. Since the main challenges of GPU architecture are the limited memory sizes, the memory transfer latency, and the vast number of GPU cores. Our system is designed to strengthen the use of GPU cores and reduce the effect of memory transfer. We propose a representation consists of indices and column-based RDF ID data that can reduce the GPU memory requirement. The indexing and pre-upload filtering techniques are then applied to reduce the data transfer between the host and GPU memory. We add the index swapping process to facilitate the sorting and joining data process based on the given variable and add the pre-upload step to reduce the size of results’ storage, and the data transfer time. The experimental results show that our representation is about 35% smaller than the traditional NT format and 40% less compared to that of gStore. The query processing time can be speedup ranging from 1.95 to 397.03 when compared with RDF3X and gStore processing time with WatDiv test suite. It achieves speedup 578.57 and 62.97 for LUBM benchmark when compared to RDF-3X and gStore. The analysis shows the query cases which can gain benefits from our approach.

Large graph convolutional network training with GPU-oriented data communication architecture

Graph Convolutional Networks (GCNs) are increasingly adopted in large-scale graph-based recommender systems. Training GCN requires the minibatch generator traversing graphs and sampling the sparsely located neighboring nodes to obtain their features. Since real-world graphs often exceed the capacity of GPU memory, current GCN training systems keep the feature table in host memory and rely on the CPU to collect sparse features before sending them to the GPUs. This approach, however, puts tremendous pressure on host memory bandwidth and the CPU. This is because the CPU needs to (1) read sparse features from memory, (2) write features into memory as a dense format, and (3) transfer the features from memory to the GPUs. In this work, we propose a novel GPU-oriented data communication approach for GCN training, where GPU threads directly access sparse features in host memory through zero-copy accesses without much CPU help. By removing the CPU gathering stage, our method significantly reduces the consumption of the host resources and data access latency. We further present two important techniques to achieve high host memory access efficiency by the GPU: (1) automatic data access address alignment to maximize PCIe packet efficiency, and (2) asynchronous zero-copy access and kernel execution to fully overlap data transfer with training. We incorporate our method into PyTorch and evaluate its effectiveness using several graphs with sizes up to 111 million nodes and 1.6 billion edges. In a multi-GPU training setup, our method is 65--92% faster than the conventional data transfer method, and can even match the performance of all-in-GPU-memory training for some graphs that fit in GPU memory.

EBV et immunodépression

Epstein-Barr virus (EBV), discovered in 1964, is a double-stranded DNA virus belonging to the Herpesviridae family. EBV has a lymphoid tropism with transforming capacities using different oncogenic viral proteins. This virus has two replication cycles: a lytic cycle mainly occuring during primary infection and a latent cycle allowing viral persistence into host memory B cells. More than 90% of adults are seropositive for EBV worldwide, with a past history of asymptomatic or mild primary infection. EBV infection can sometimes cause life-threatening complications such as hemophagocytic lymphohistiocytosis, and lead to the development of lymphoproliferative disorders or cancers. Risk factors associated with these phenotypes have been recently described through the study of monogenic primary immune deficiencies with EBV susceptibility. We here review the virological and immunological aspects of EBV infection and EBV-related complications with an overview of current available treatments.

Mille Cheval: a GPU-based in-memory high-performance computing framework for accelerated processing of big-data streams

Streams are temporally ordered, rapid changing, ample in volume, and infinite in nature. It is nearly impossible to store the entire data stream due to its large volume and high velocity. In this work, the principle of parallelism is employed to accelerate stream data computing. GPU-based high-performance computing (HPC) framework is proposed for accelerated processing of big-data streams using the in-memory data structure. We have implemented three parallel algorithms to prove the viability of the framework. The contributions of Mille Cheval are: (1) the viability of streaming on accelerators to increase throughput, (2) carefully chosen hash algorithms to achieve low collision rate and high randomness, and (3) memory sketches for approximation. The objective is to leverage the power of a single node using in-memory computing and hybrid computing. HPC does not always require high-end hardware but well-designed algorithms. Achievements of Mille Cheval are: (1) relative error is 1.32 when error rate and overestimate rate are chosen as 0.001 and (2) the host memory space requirement is just 63 MB for 1 terabyte of data. The proposed algorithms are pragmatic. It is evident from experimental results that the framework demonstrates 10X speed-up as compared with CPU implementations and 3X speed-up as compared with GPU implementations.

Efficient Flow Processing in 5G-Envisioned SDN-Based Internet of Vehicles Using GPUs

In the 5G-envisioned Internet of vehicles (IoV), a significant volume of data is exchanged through networks between intelligent transport systems (ITS) and clouds or fogs. With the introduction of Software-Defined Networking (SDN), the problems mentioned above are resolved by high-speed flow-based processing of data in network systems. To classify flows of packets in the SDN network, high throughput packet classification systems are needed. Although software packet classifiers are cheaper and more flexible than hardware classifiers, they could only deliver limited performance. A key idea to resolve this problem is parallelizing packet classification on graphical processing units (GPUs). In this paper, we study parallel forms of Tuple Space Search and Pruned Tuple Space Search algorithms for the flow classification suitable for GPUs using CUDA (Compute Unified Device Architecture). The key idea behind the offered methodology is to transfer the stream of packets from host memory to the global memory of the CUDA device, then assigning each of them to a classifier thread. To evaluate the proposed method, the GPU-based versions of the algorithms were implemented on two different CUDA devices, and two different CPU-based implementations of the algorithms were used as references. Experimental results showed that GPU computing enhances the performance of Pruned Tuple Space Search remarkably more than Tuple Space Search. Moreover, results evinced the computational efficiency of the proposed method for parallelizing packet classification algorithms.

Improved data transfer efficiency for scale‐out heterogeneous workloads using on‐the‐fly I/O link compression

SummaryGraphics processing units (GPUs) are unarguably vital to keep up with the perpetually growing demand for compute capacity of data‐intensive applications. However, the overhead of transferring data between host and GPU memory is already a major limiting factor on the single‐node level. The situation intensifies in scale‐out scenarios, where data movement is becoming even more expensive. By augmenting the CloudCL framework with 842‐based compression facilities, this article demonstrates that transparent on‐the‐fly I/O link compression can yield performance improvements between 1.11× and 2.07× across tested scale‐out GPU workloads.

EMOGI

Modern analytics and recommendation systems are increasingly based on graph data that capture the relations between entities being analyzed. Practical graphs come in huge sizes, offer massive parallelism, and are stored in sparse-matrix formats such as compressed sparse row (CSR). To exploit the massive parallelism, developers are increasingly interested in using GPUs for graph traversal. However, due to their sizes, graphs often do not fit into the GPU memory. Prior works have either used input data pre-processing/partitioning or unified virtual memory (UVM) to migrate chunks of data from the host memory to the GPU memory. However, the large, multi-dimensional, and sparse nature of graph data presents a major challenge to these schemes and results in significant amplification of data movement and reduced effective data throughput. In this work, we propose EMOGI, an alternative approach to traverse graphs that do not fit in GPU memory using direct cache-line-sized access to data stored in host memory. This paper addresses the open question of whether a sufficiently large number of overlapping cache-line-sized accesses can be sustained to 1) tolerate the long latency to host memory, 2) fully utilize the available bandwidth, and 3) achieve favorable execution performance. We analyze the data access patterns of several graph traversal applications in GPU over PCIe using an FPGA to understand the cause of poor external bandwidth utilization. By carefully coalescing and aligning external memory requests, we show that we can minimize the number of PCIe transactions and nearly fully utilize the PCIe bandwidth with direct cache-line accesses to the host memory. EMOGI achieves 2.60X speedup on average compared to the optimized UVM implementations in various graph traversal applications. We also show that EMOGI scales better than a UVM-based solution when the system uses higher bandwidth interconnects such as PCIe 4.0.

Comparing unified, pinned, and host/device memory allocations for memory‐intensive workloads on Tegra SoC

SummaryEdge computing focuses on processing near the source of the data. Edge computing devices using the Tegra SoC architecture provide a physically distinct GPU memory architecture. In order to take advantage of this architecture, different modes of memory allocation need to be considered. Different GPU memory allocation techniques yield different results in memory usage and execution times of identical applications on Tegra devices. In this article, we implement several GPU application benchmarks, including our custom CFD code with unified, pinned, and normal host/device memory allocation modes. We evaluate and compare the memory usage and execution time of such workloads on edge computing Tegra system‐on‐chips (SoC) equipped with integrated GPUs using a shared memory architecture, and non‐SoC machines with discrete GPUs equipped with distinct VRAM. We discover that utilizing normal memory allocation methods on SoCs actually use double the required memory because of unnecessary device memory copies, despite being physically shared with host memory. We show that GPU application memory usage can be reduced up to 50%, and that even performance improvements can occur just by replacing normal memory allocation and memory copy methods with managed unified memory or pinned memory allocation.

Kidney-intrinsic factors determine the severity of ischemia/reperfusion injury in a mouse model of delayed graft function

Delayed graft function due to transplant ischemia/reperfusion injury adversely affects up to 50% of deceased-donor kidney transplant recipients. However, key factors contributing to the severity of ischemia/reperfusion injury remain unclear. Here, using a clinically relevant mouse model of delayed graft function, we demonstrated that donor genetic background and kidney-intrinsic MyD88/Trif-dependent innate immunity were key determinants of delayed graft function. Functional deterioration of kidney grafts directly corresponded with the duration of cold ischemia time. The graft dysfunction became irreversible after cold ischemia time exceeded six hours. When cold ischemia time reached four hours, kidney grafts displayed histological features reflective of delayed graft function seen in clinical kidney transplantation. Notably, kidneys of B6 mice exhibited significantly more severe histological and functional impairment than kidneys of C3H or BALB/c mice, regardless of recipient strains or alloreactivities. Furthermore, allografts of B6 mice also showed an upregulation of IL-6, neutrophil gelatinase-associated lipocalin, and endoplasmic reticulum stress genes, as well as an increased influx of host neutrophils and memory CD8 T-cells. In contrast, donor MyD88/Trif deficiency inhibited neutrophil influx and decreased the expression of IL-6 and endoplasmic reticulum stress genes, along with improved graft function and prolonged allograft survival. Thus, kidney-intrinsic factors involving genetic characteristics and innate immunity serve as critical determinants of the severity of delayed graft function. This preclinical murine model allows for further investigations of the mechanisms underlying delayed graft function.

Cascaded Write Amplification of LSM-tree-based Key-Value Stores underlying Solid-State Disks

Log-structured merge tree (i.e., LSM-tree)-based key–value stores (i.e., KV stores) are widely used in big-data applications and provide high performance. NAND Flash-based Solid-state disks (i.e., SSDs) have become a popular storage device alternative to hard disk drives (i.e., HDDs) because of their high performance and low power consumption. LSM-tree KV stores with SSDs are deployed in large-scale storage systems, which aims to achieve high performance in the cloud. Write amplification in LSM-tree KV stores and NAND Flash memory in SSDs are defined as WA1 and WA2 in this paper. The former, which is attributed to compaction operations in LSM-tree-based KV stores, is a burden on I/O bandwidth between the host and the device. The latter, which results from out-place updates in NAND Flash memory, blocks user I/O requests between the host and NAND Flash memory, thereby degrading the SSD performance. Write amplification impairs the overall system performance. In this study, we explored the two-level cascaded write amplification in LSM-tree KV stores with SSDs. The cascaded write amplification is represented as WA. Our primary goal is to comprehensively study two-level cascaded write amplification on the host-side LSM-tree KV stores and the device-side SSDs. We quantitatively analyze the impact of two-level write amplification on overall performance. The cascaded write amplification is 16.44 (WA1 is 16.55; WA2 is 0.99) and 35.51 (WA1 is 16.6; WA2 is 2.14) for SSD-I and SSD-S with LevelDB’s default setting under DB_bench. The larger cascaded write amplification of KV stores has a bad impact on SSD performance and lifetime. The throughput of SSD-S and SSD-I under an 80%-write workload is approximately 0.28x and 0.31x of that under a 100%-write workload. Therefore, it is important to design a novel approach to balance the cost of an SSD lifetime caused by cascaded write amplification and its high performance under the read-write-mixed workloads. We attempt to reveal details of cascaded write amplification and hope that this study is useful for developers of LSM-tree-based KV stores and SSD software stacks.

HMB-I/O: Fast Track for Handling Urgent I/Os in Nonvolatile Memory Express Solid-State Drives

Differentiated I/O services for applications with their own requirements are very important for user satisfaction. Nonvolatile memory express (NVMe) solid-state drive (SSD) architecture can improve the I/O bandwidth with its numerous submission queues, but the quality of service (QoS) of each I/O request is never guaranteed. In particular, if many I/O requests are pending in the submission queues due to a bursty I/O workload, urgent I/O requests can be delayed, and consequently, the QoS requirements of applications that need fast service cannot be met. This paper presents a scheme that handles urgent I/O requests without delay even if there are many pending I/O requests. Since the pending I/O requests in the submission queues cannot be controlled by the host, the host memory buffer (HMB), which is part of the DRAM of the host that can be accessed from the controller, is used to process urgent I/O requests. Instead of sending urgent I/O requests into the SSDs through legacy I/O paths, the latency is removed by directly inserting them into the HMB. Emulator experiments demonstrated that the proposed scheme could reduce the average and tail latencies by up to 99% and 86%, respectively.

Hierarchical Orchestration of Disaggregated Memory

This article presents XMemPod , a hierarchical disaggregated memory orchestration system. XMemPod virtualizes cluster wide memory to scale large memory workloads in virtualized clouds. It makes three novel contributions: (1) XMemPod offers efficient, transparent, and dynamic sharing of available memory that is disaggregated across VMs on the same host or in the cluster. (2) XMemPod provides a hierarchical memory expansion framework, which enables memory-intensive workloads on a VM to expand its memory demand over virtualized host memory first, and remote memory next, before resorting to external disk. (3) XMemPod provides a suite of optimization techniques to further improve the utilization and access latency of disaggregated memory. XMemPod is deployed on a virtualized RDMA cluster without any modifications to user applications and the OSes. Evaluated with multiple workloads on unmodified Spark, Apache Hadoop, Memcached, Redis and VoltDB, using XMemPod , throughputs of these applications improve by 11x to 612x over conventional Linux, and by 1.7x to 14x over the existing representative remote memory paging systems, and yet the total amount of network traffic consumed by XMemPod is only 24 percent of the existing approaches.

Communication protocol optimization for enhanced GPU performance

The U.S. Department of Energy CORAL program systems SUMMIT and SIERRA are based on hybrid servers comprising IBM POWER9 CPUs and NVIDIA V100 graphics processing units (GPUs) connected by two extended data rate (EDR) links to a high-speed InfiniBand Network. A major challenge to the communication software stack is to optimize performance for all combinations of data origin and destination: host or GPU memory, same or different server. Alternate paths exist for routing data from GPU memory. When origin and destination are on different servers, it can be sent either via host memory or bypassing host memory with GPU direct feature. When origin and destination are on the same server, host memory can be bypassed with peer-to-peer inter process communication (IPC). For large messages pipelining makes host memory data path competitive with GPU direct. In this article, we explain the techniques used in Spectrum MPI Parallel Active Message Interface layer to cache memory types and attributes in order to reduce the overhead associated with calling the CUDA application programming interface (API); in addition, we detail the different protocols used for different memory types, device memory, managed memory, and host memory. To illustrate, the caching technique achieved a device-to-device latency improvement of 26% for intranode transfers and 19% for internode transfers.

HMB in DRAM-less NVMe SSDs: Their usage and effects on performance.

Solid-state drives (SSDs) that do not have internal dynamic random-access memory (DRAM) are being widely spread for client SSD and embedded SSD markets in recent years because they are cheap and consume less power. Obviously, their performance is lower than conventional SSDs because they cannot exploit advantages of DRAM in the controller. However, this problem can be alleviated by using host memory buffer (HMB) feature of Non-Volatile Memory Express (NVMe), which allows SSDs to utilize the DRAM of host. In this paper, we show that commercial DRAM-less SSDs clearly exhibit worse I/O performance than SSDs with internal DRAM, but this can be improved by using the HMB feature. We also present methods that reveal how the host memory buffer is used in commercial DRAM-less SSDs to improve I/O performance. Through extensive experiments, we conclude that DRAM-less SSDs evaluated in this study mainly exploit the host memory buffer as an address mapping table cache rather than a read cache or write buffer to improve I/O performance.

Traversing large graphs on GPUs with unified memory

Due to the limited capacity of GPU memory, the majority of prior work on graph applications on GPUs has been restricted to graphs of modest sizes that fit in memory. Recent hardware and software advances make it possible to address much larger host memory transparently as a part of a feature known as unified virtual memory. While accessing host memory over an interconnect is understandably slower, the problem space has not been sufficiently explored in the context of a challenging workload with low computational intensity and an irregular data access pattern such as graph traversal. We analyse the performance of breadth first search (BFS) for several large graphs in the context of unified memory and identify the key factors that contribute to slowdowns. Next, we propose a lightweight offline graph reordering algorithm, HALO (Harmonic Locality Ordering), that can be used as a pre-processing step for static graphs. HALO yields speedups of 1.5x-1.9x over baseline in subsequent traversals. Our method specifically aims to cover large directed real world graphs in addition to undirected graphs whereas prior methods only account for the latter. Additionally, we demonstrate ties between the locality ordering problem and graph compression and show that prior methods from graph compression such as recursive graph bisection can be suitably adapted to this problem.

High-performance attribute reduction on graphics processing unit

ABSTRACT Recently, graphics processing unit (GPU) gained lots of attention from academia and industry for its applicability in high-performance computing. It has been successfully applied to many fields, such as image processing, machine learning, object detection, etc. In our previous work, GPU was adopted to accelerate the computation of rough set approximation (RSA), which is the core step in most of the rough sets based tasks, e.g. attribute reduction. The method is essentially a CPU-GPU cooperative paradigm. That is to say, there are lots of data exchanged between host memory and GPU memory, which greatly degrades the performance of the system. This paper introduces a unified GPU framework for parallel attribute reduction, in which two critical steps in attribute reduction, i.e. computation of equivalence class and attributes significance, are both executed on GPU. Moreover, the algorithm is well designed by exploiting the architectural characteristics of the modern GPU architecture. Experiments were carried out on data sets with different sizes. The results show that the proposed algorithm can outperform the CPU-GPU cooperative algorithm on large data sets.

Lack of B Lymphocytes Enhances CD8 T Cell-Mediated Resistance against Respiratory Viral Infection but Compromises Memory Cell Formation.

Following a respiratory virus infection, CXCR3hi CX3CR1lo and CXCR3lo CX3CR1hi CD8 T cells localize to different compartments within the lung and play an important role in host resistance, but mechanisms governing their optimal generation are poorly defined. We serendipitously found that B cell-deficient (μMT-/-) mice were highly resistant to lethal infection with a virulent poxvirus strain and that depletion of CD8 T cells rendered these mice susceptible to infection. B cells were not required for the expansion of virus-specific CD8 T cells, but a greater proportion of activated CD8 T cells acquired an effector-like CXCR3lo CX3CR1hi phenotype in the absence of B cells. After recovery from infection, CD8 T cells in μMT-/- mice contracted normally but failed to survive and seed the memory cell pool in both the lungs and spleen. These findings reveal a previously unappreciated role for B cells in regulating the balance between CD8 T cell-mediated resistance against respiratory viral infection and memory cell development.IMPORTANCE B cells play critical role in host resistance against many respiratory viral infections. However, the role of B cells beyond antibody-producing cells is less well defined. In this study, we made a surprising observation that mice lacking B cells were more resistant to respiratory infection with vaccinia virus than wild-type mice. This enhanced resistance was mediated by CD8 T cells because when we depleted CD8 T cells in B cell-deficient mice, these mice were unable to survive the infection. Interestingly, CD8 T cells in B cell-deficient mice were skewed more toward effector phenotype and less toward memory phenotype, which resulted in severely compromised memory CD8 T cell development. Thus, our study shows a novel role of B cells as regulators of CD8 T cell-mediated host resistance and memory CD8 T cell formation during respiratory viral infection.