GPU Communication Research Articles

Real-time rendering and physics of complex dynamic terrains modeled as CSG trees of DEMs carved with spheres

We present a novel proposal for modeling complex dynamic terrains that offers real-time rendering, dynamic updates and physical interaction of entities simultaneously. We can capture any feature from landscapes including tunnels, overhangs and caves, and we can conduct a total destruction of the terrain. Our approach is based on a Constructive Solid Geometry tree, where a set of spheres are subtracted from a base Digital Elevation Model. Erosions on terrain are easily and efficiently carried out with a spherical sculpting tool with pixel-perfect accuracy. Real-time rendering performance is achieved by applying a one-direction CPU–GPU communication strategy and using the standard depth and stencil buffer functionalities provided by any graphics processor.

Large-scale flow simulations using lattice Boltzmann method with AMR following free-surface on multiple GPUs

Free-surface flow simulations require high-resolution grids to capture phenomena at the interface as well as a long computational time. In this paper, we propose a numerical method for realizing large-scale free-surface flow simulations using the lattice Boltzmann method and multiple GPUs. By introducing the adaptive mesh refinement (AMR) method, which adapts high-resolution grids to free surfaces, to the lattice Boltzmann method, the number of lattice points can be greatly reduced. In the calculation of the AMR method, the spatial distribution of a computational load changes with time; therefore, the number of lattice points assigned to each GPU is kept equal by dynamic domain partitioning using a space-filling curve. We measured the weak scalability of our AMR code on the TSUBAME3.0 supercomputer at the Tokyo Institute of Technology. By hiding GPU–GPU communication overheads by the overlapping method, the performance increased 1.29 times that of the naïve implementation, and we achieved the fairly high performance of 14,570 MLUPS using 256 GPUs. We demonstrate large-scale simulations for the dam breaking problem and show a reduction in computational cost with the AMR method.

Towards extreme scale dissipative particle dynamics simulations using multiple GPGPUs

A multi-GPGPU development for Mesoscale Simulations using the Dissipative Particle Dynamics method is presented. This distributed GPU acceleration development is an extension of the DL_MESO package to MPI+CUDA in order to exploit the computational power of the latest NVIDIA cards on hybrid CPU–GPU architectures. Details about the extensively applicable algorithm implementation and memory coalescing data structures are presented. The key algorithms’ optimizations for the nearest-neighbour list searching of particle pairs for short range forces, exchange of data and overlapping between computation and communications are also given. We have carried out strong and weak scaling performance analyses with up to 4096 GPUs. A two phase mixture separation test case with 1.8 billion particles has been run on the Piz Daint supercomputer from the Swiss National Supercomputer Center. With CUDA aware MPI, proper GPU affinity, communication and computation overlap optimizations for multi-GPU version, the final optimization results demonstrated more than 94% efficiency for weak scaling and more than 80% efficiency for strong scaling. As far as we know, this is the first report in the literature of DPD simulations being run on this large number of GPUs. The remaining challenges and future work are also discussed at the end of the paper.

Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect

High performance multi-GPU computing becomes an inevitable trend due to the ever-increasing demand on computation capability in emerging domains such as deep learning, big data and planet-scale simulations. However, the lack of deep understanding on how modern GPUs can be connected and the real impact of state-of-the-art interconnect technology on multi-GPU application performance become a hurdle. In this paper, we fill the gap by conducting a thorough evaluation on five latest types of modern GPU interconnects: PCIe, NVLink-V1, NVLink-V2, NVLink-SLI and NVSwitch, from six high-end servers and HPC platforms: NVIDIA P100-DGX-1, V100-DGX-1, DGX-2, OLCF's SummitDev and Summit supercomputers, as well as an SLI-linked system with two NVIDIA Turing RTX-2080 GPUs. Based on the empirical evaluation, we have observed four new types of GPU communication network NUMA effects: three are triggered by NVLink's topology, connectivity and routing, while one is caused by PCIe chipset design issue. These observations indicate that, for an application running in a multi-GPU node, choosing the right GPU combination can impose considerable impact on GPU communication efficiency, as well as the application's overall performance. Our evaluation can be leveraged in building practical multi-GPU performance models, which are vital for GPU task allocation, scheduling and migration in a shared environment (e.g., AI cloud and HPC centers), as well as communication-oriented performance tuning.

GPU-powered, interactive flow simulation on a peer-to-peer group of mobile devices

This article develops novel application software which implements interactive, GPU-powered flow simulation on a group of wirelessly-connected mobile devices. Interactive simulation is an emerging field in engineering with use cases appearing in design, analysis and communication. Herein, we present a new Android-based, interactive flow solver capable of running on a wider range of multiple, wirelessly-connected mobile GPUs. The software consists of a 2D Lattice-Boltzmann Method flow physics solver, implemented using OpenGL ES 3.2, as well as a communication library which uses Wi-Fi Direct to communicate between connected devices. We compare the performance of the OpenGL-based solver against existing implementations in CUDA and demonstrate similar computational throughput. We also test a variety of communication strategies based on configurations of GPU memory mapping and communication frequency. Results confirm that passing large amounts of data infrequently offers the best overall efficiency. However, due to the extended time required to pass larger amounts of data to adjacent devices, this configuration can introduce an undesirable stuttering in an interactive application. Finally, comparisons between two and three device networks to the serial case show that, despite the inevitable cost of communication, it is possible to maintain an interactive frame rate across multiple devices; the extension of calculations across multiple devices in this way, allows the tackling of problems which are larger and of higher-resolution that previous.

Design considerations for GPU‐aware collective communications in MPI

SummaryGPU accelerators have established themselves in the state‐of‐the‐art clusters by offering high performance and energy efficiency. In such systems, efficient inter‐process GPU communication is of paramount importance to application performance. This paper investigates various algorithms in conjunction with the latest GPU features to improve GPU collective operations. First, we propose a GPU Shared Buffer‐aware (GSB) algorithm and a Binomial Tree Based (BTB) algorithm for GPU collectives on single‐GPU nodes. We then propose a hierarchical framework for clusters with multi‐GPU nodes. By studying various combinations of algorithms, we highlight the importance of choosing the right algorithm within each level. The evaluation of our framework on MPI_Allreduce shows promising performance results for large message sizes.To address the shortcoming for small and medium messages, we present the benefit of using the Hyper‐Q feature and the MPS service in jointly using CUDA IPC and host‐staged copy types to perform multiple inter‐process communications. However, we argue that efficient designs are still required to further harness this potential. Accordingly, we propose a static and a dynamic algorithm for MPI_Allgather and MPI_Allreduce and present their effectiveness on various message sizes. Our profiling results indicate that the achieved performance is mainly rooted in overlapping different copy types.

On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8x reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25% savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

Exploiting heterogeneity of communication channels for efficient GPU selection on multi-GPU nodes

Abstract Multi-GPU nodes have become the platform of choice for scientific applications. In a multi-GPU node, GPUs are interconnected together via different communication channels. The intranode communications among GPUs may traverse different paths with different latency and bandwidth characteristics. As the number of GPUs within a multi-GPU node increases, the physical topology of the GPU interconnects tend to have more levels of hierarchy, which in turn increases the heterogeneity of the GPU communication channels.In this paper, we show that the performance of different intranode GPU communication channels can be considerably different from each other. Accordingly, we propose a topology-aware GPU selection scheme for efficient assignment of GPUs to the MPI processes within a node. The resulting assignment helps to improve the communication performance by mapping more intensive inter-process GPU-to-GPU communications on the stronger communication channels. We leverage three metrics in our scheme to distinguish among different GPU-to-GPU communication channels: latency, bandwidth, and distance. We evaluate our scheme through extensive experiments conducted on a 16-GPU node, and show that our scheme can provide considerable performance improvements over the default GPU selection scheme. In particular, we can achieve up to 70% and 21% performance improvement at the microbenchmark and application level, respectively.

A Framework for Direct and Transparent Data Exchange of Filter-stream Applications in Multi-GPUs Architectures

The massive data generation has been pushing for significant advances in computing architectures, reflecting in heterogeneous architectures composed by different types of processing units. The filter-stream paradigm is typically used to exploit the parallel processing power of these new architectures. The efficiency of applications in this paradigm is achieved by exploring a set of interconnected computers (cluster) using filters and communication between them in a coordinated way. In this work we propose, implement and test a generic abstraction for direct and transparent data exchange of filter-stream applications in heterogeneous cluster with multi-GPU (Graphics Processing Units). This abstraction allows hiding from the programmers all the low-level implementation details related to GPU communication and the control related to the location of filters. Further, we consolidate such abstraction into a framework. Empirical assessments using a real application show that the proposed abstraction layer eases the implementation of filter-stream applications without compromising the overall application performance.

TOD-Tree: Task-Overlapped Direct Send Tree Image Compositing for Hybrid MPI Parallelism and GPUs.

Modern supercomputers have thousands of nodes, each with CPUs and/or GPUs capable of several teraflops. However, the network connecting these nodes is relatively slow, on the order of gigabits per second. For time-critical workloads such as interactive visualization, the bottleneck is no longer computation but communication. In this paper, we present an image compositing algorithm that works on both CPU-only and GPU-accelerated supercomputers and focuses on communication avoidance and overlapping communication with computation at the expense of evenly balancing the workload. The algorithm has three stages: a parallel direct send stage, followed by a tree compositing stage and a gather stage. We compare our algorithm with radix-k and binary-swap from the IceT library in a hybrid OpenMP/MPI setting on the Stampede and Edison supercomputers, show strong scaling results and explain how we generally achieve better performance than these two algorithms. We developed a GPU-based image compositing algorithm where we use CUDA kernels for computation and GPU Direct RDMA for inter-node GPU communication. We tested the algorithm on the Piz Daint GPU-accelerated supercomputer and show that we achieve performance on par with CPUs. Last, we introduce a workflow in which both rendering and compositing are done on the GPU.

On Optimal Kernel Size for Integrated CPU-GPUs — A Case Study

Integrated CPU-GPU architectures with a fully addressable shared memory completely eliminate any CPU-GPU data transfer overhead. Since such architectures are relatively new, it is unclear what level of interaction between the CPU and GPU attains the best energy efficiency. Too coarse grained or larger kernels with fairly low CPU - GPU interaction could cause poor utilization of the shared resources while too fine grained kernels could cause frequent interrupts of GPU computation and performance degradation. Also larger kernels require larger shared resources causing increase in area and parasitics which affect the latency sensitive CPU cores. In this paper, we show the effect of granularity on the overall system's energy efficiency using a synthetic workload. We describe how our framework models a truly unified shared memory in integrated architectures with frequent CPU - GPU communication.

A run-time optimization approach for reducing data movements using locality-aware searching

The CPU---GPU communication bottleneck limits the performance improvement of GPU applications in heterogeneous GPGPU systems and usually is handled by data reuse optimization. This paper analyzes data reuse through DAG abstraction and obtains rules showing that the run-time data reuse optimization can effectively relieve the bottleneck. Based on the rules, this paper proposes a run-time optimization framework for data reuse, called R-Tracker. The R-Tracker uses locality-aware searching approach to handle reuses. It can not only low costly implement the data reuse optimization but also effectively implement the searching, the data transfers, and the GPU computation concurrently. R-Tracker relaxes the constraints that are required in compiler-based approaches and thus achieves better reuse effect. The experimental results show that R-Tracker improves the performance by 1.77---16.42 % over compiler-based approach OpenMPC and 1.40---8.39 % over CGCM in single-node execution, and 48.78---60 % over CGCM in multi-node execution.

Combining multi-core and GPU computing for solving combinatorial optimization problems

In this paper, we revisit the design and implementation of Branch-and-Bound (B&B) algorithms for solving large combinatorial optimization problems on GPU-enhanced multi-core machines. B&B is a tree-based optimization method that uses four operators (selection, branching, bounding and pruning) to build and explore a highly irregular tree representing the solution space. In our previous works, we have proposed a GPU-accelerated approach in which only a single CPU core is used and only the bounding operator is performed on the GPU device. Here, we extend the approach (LL-GB&B) in order to minimize the CPU–GPU communication latency and thread divergence. Such an objective is achieved through a GPU-based fine-grained parallelization of the branching and pruning operators in addition to the bounding one. The second contribution consists in investigating the combination of a GPU with multi-core processing. Two scenarios have been explored leading to two approaches: a concurrent (RLL-GB&B) and a cooperative one (PLL-GB&B). In the first one, the exploration process is performed concurrently by the GPU and the CPU cores. In the cooperative approach, the CPU cores prepare and off-load to GPU pools of tree nodes using data streaming while the GPU performs the exploration. The different approaches have been extensively experimented on the Flowshop scheduling problem. Compared to a single CPU-based execution, LL-GB&B allows accelerations up to (×160) for large problem instances. Moreover, when combining multi-core and GPU, we figure out that using RLL-GB&B is not beneficial while PLL-GB&B enables an improvement up to 36% compared to LL-GB&B.

Optimizing Linpack Benchmark on GPU-Accelerated Petascale Supercomputer

In this paper we present the programming of the Linpack benchmark on TianHe-1 system, the first petascale supercomputer system of China, and the largest GPU-accelerated heterogeneous system ever attempted before. A hybrid programming model consisting of MPI, OpenMP and streaming computing is described to explore the task parallel, thread parallel and data parallel of the Linpack. We explain how we optimized the load distribution across the CPUs and CPUs using the two-level adaptive method and describe the implementation in details. To overcome the low-bandwidth between the CPU and GPU communication, we present a software pipelining technique to hide the communication overhead. Combined with other traditional optimizations, the Linpack we developed achieved 196.7 GFLOPS on a single compute element of TianHe-1. This result is 70.1% of the peak compute capability, 3.3 times faster than the result by using the vendor's library. On the full configuration of TianHe-1 our optimizations resulted in a Linpack performance of 0.563 PFLOPS, which made TianHe-1 the 5th fastest supercomputer on the Top500 list in November, 2009.

The development of Mellanox/NVIDIA GPUDirect over InfiniBand—a new model for GPU to GPU communications

The usage and adoption of General Purpose GPUs (GPGPU) in HPC systems is increasing due to the unparalleled performance advantage of the GPUs and the ability to fulfill the ever-increasing demands for floating points operations. While the GPU can offload many of the application parallel computations, the system architecture of a GPU-CPU-InfiniBand server does require the CPU to initiate and manage memory transfers between remote GPUs via the high speed InfiniBand network. In this paper we introduce for the first time a new innovative technology--GPUDirect that enables Tesla GPUs to transfer data via InfiniBand without the involvement of the CPU or buffer copies, hence dramatically reducing the GPU communication time and increasing overall system performance and efficiency. We also explore for the first time the performance benefits of GPUDirect using Amber and LAMMPS applications.