multi-GPU Systems Research Articles

Analysis of application of distributed multi-node, multi-GPU heterogeneous system for acceleration of image reconstruction in Electrical Capacitance Tomography

Abstract 3D ECT provides a lot of challenging computational issues that have been reported in the past by many researchers. Image reconstruction using deterministic methods requires execution of many basic operations of linear algebra, such as matrix transposition, multiplication, addition and subtraction. In order to reach real-time reconstruction a 3D ECT computational subsystem has to be able to transform capacitance data into image in fractions of seconds. By assuming, that many of the computations can be performed in parallel using modern, fast graphics processor and by altering the algorithms time to achieve high quality image reconstruction will be shortened significantly. The research conducted while analysing ECT algorithms has also shown that, although dynamic development of GPU computational capabilities and its recent application for image reconstruction in ECT has significantly improved calculations time, in modern systems a single GPU is not enough to perform many tasks. Distributed Multi-GPU solutions can reduce reconstruction time to only a fraction of what was possible on pure CPU systems. Nevertheless performed tests clearly illustrate the need for developing a new distributed platform, which would be able to fully utilize the potential of the hardware. It has to take into account specific nature of computations in Multi-GPU systems.

Global magnetohydrodynamic simulations on multiple GPUs

Global magnetohydrodynamic (MHD) models play the major role in investigating the solar wind–magnetosphere interaction. However, the huge computation requirement in global MHD simulations is also the main problem that needs to be solved. With the recent development of modern graphics processing units (GPUs) and the Compute Unified Device Architecture (CUDA), it is possible to perform global MHD simulations in a more efficient manner. In this paper, we present a global magnetohydrodynamic (MHD) simulator on multiple GPUs using CUDA 4.0 with GPUDirect 2.0. Our implementation is based on the modified leapfrog scheme, which is a combination of the leapfrog scheme and the two-step Lax–Wendroff scheme. GPUDirect 2.0 is used in our implementation to drive multiple GPUs. All data transferring and kernel processing are managed with CUDA 4.0 API instead of using MPI or OpenMP. Performance measurements are made on a multi-GPU system with eight NVIDIA Tesla M2050 (Fermi architecture) graphics cards. These measurements show that our multi-GPU implementation achieves a peak performance of 97.36 GFLOPS in double precision.

GPU를 이용한 기타의 음 합성을 위한 효과적인 병렬 구현

본 논문에서는GPU 환경에서 기타의 음합성을 위한 물리적 모델링의 효율적인 병렬구현 방법을 제안한다. 물리적 모델링을 이용하여 기타의 개방현(E2, A2, D3, G4, B3, E4)들의 기본음을 합성하기 위해 각 개방현 음 합성을 위한 적절한 필터 계수를 사용하였고, 지연 라인의 길이를 조절하였다. 또한 물리적 모델링 알고리즘을 분석한 결과 지연 라인의 길이만큼 병렬성을 갖는 것을 확인하였다. 따라서 각 개방현의 기타 음을 합성하기 위해 지연 라인의 길이만큼CUDA 코어를 할당한 후 최적의 성능을 보이도록 알고리즘을 병렬 구현하였다. 모의실험결과, GPU를 이용하여 합성한 기타 음과 원음과의 스펙트럼이 매우 유사하였고, GPU는 기존 고성능 TI DSP보다 68배, CPU보다 3배의 성능 향상을 보였다. 또한, 본 논문에서는 물리적 모델링 알고리즘을 멀티 GPU시스템에서도 구현하고 성능을 분석하였다. This paper proposes an effective parallel implementation of a physical modeling synthesis of guitar on the GPU environment. We used appropriate filter coefficients and adjusted the length of delay line for each open string to generate 44,100 six-polyphonic guitar sounds (E2, A2, D3, G4, B3, E4) by using physical modeling synthesis. In addition, we analyzed the physical modeling synthesis algorithm and observed that we can exploit parallelism inherent in the length of delay line. Thus, we assigned CUDA cores as many as the length of delay line and effectively implemented the physical modeling synthesis using GPU to achieve the highest performance. Experimental results indicated that synthetic guitar sounds using GPU were very similar to the original sounds when we compared their spectra. In addition, GPU achieved 68x and 3x better performance than high-performance TI DSP and CPU, respectively. Furthermore, this paper implemented and evaluated the performance of multi-GPU systems for the physical modeling algorithm.

Dynamic load balancing on heterogeneous multi-GPU systems

Actual HPC systems are composed by multicore processors and powerful graphics processing units. Adapting existing code and libraries to these new systems is a fundamental problem due to the important increment on programming difficulties. The heterogeneity, both at architectural and programming levels at the same time, raises the programmability wall. The performance of the code is affected by the large interdependence between the code and the parallel architecture. We have developed a dynamic load balancing library that allows parallel code to be adapted to a wide variety of heterogeneous systems. The overhead introduced by our system is minimal and the cost to the programmer negligible. This system has been successfully applied to solve load imbalance problems appearing in homogeneous and heterogeneous multiGPU platforms. We consider the Dynamic Programming technique as case of study to validate our proposals using different heterogeneous scenarios in multiGPU systems.

Introducing and Implementing the Allpairs Skeleton for Programming Multi-GPU Systems

Algorithmic skeletons simplify software development: they abstract typical patterns of parallelism and provide their efficient implementations, allowing the application developer to focus on the structure of algorithms, rather than on implementation details. This becomes especially important for modern parallel systems with multiple graphics processing units (GPUs) whose programming is complex and error-prone, because state-of-the-art programming approaches like CUDA and OpenCL lack high-level abstractions. We define a new algorithmic skeleton for allpairs computations which occur in real-world applications, ranging from bioinformatics to physics. We develop the skeleton’s generic parallel implementation for multi-GPU Systems in OpenCL. To enable the automatic use of the fast GPU memory, we identify and implement an optimized version of the allpairs skeleton with a customizing function that follows a certain memory access pattern. We use matrix multiplication as an application study for the allpairs skeleton and its two implementations and demonstrate that the skeleton greatly simplifies programming, saving up to 90 % of lines of code as compared to OpenCL. The performance of our optimized implementation is up to 6.8 times higher as compared with the generic implementation and is competitive to the performance of a manually written optimized OpenCL code.

Scalable Parallel Motion Estimation on Muti-GPU System

With NVIDIA’s parallel computing architecture CUDA, using GPU to speed up compute-intensive applications has become a research focus in recent years. In this paper, we proposed a scalable method for multi-GPU system to accelerate motion estimation algorithm, which is the most time consuming process in video encoding. Based on the analysis of data dependency and multi-GPU architecture, a parallel computing model and a communication model are designed. We tested our parallel algorithm and analyzed the performance with 10 standard video sequences in different resolutions using 4 NVIDIA GTX460 GPUs, and calculated the overall speedup. Our results show that a speedup of 36.1 times using 1 GPU and more than 120 times for 4 GPUs on 1920x1080 sequences. Further, our parallel algorithm demonstrated the potential of nearly linear speedup according to the number of GPUs in the system.

Efficient breadth first search on multi-GPU systems

Simple algorithms for the execution of a Breadth First Search on large graphs lead, running on clusters of GPUs, to a situation of load unbalance among threads and un-coalesced memory accesses, resulting in pretty low performances. To obtain a significant improvement on a single GPU and to scale by using multiple GPUs, we resort to a suitable combination of operations to rearrange data before processing them. We propose a novel technique for mapping threads to data that achieves a perfect load balance by leveraging prefix-sum and binary search operations. To reduce the communication overhead, we perform a pruning operation on the set of edges that needs to be exchanged at each BFS level. The result is an algorithm that exploits at its best the parallelism available on a single GPU and minimizes communication among GPUs. We show that a cluster of GPUs can efficiently perform a distributed BFS on graphs with billions of nodes.

SU-E-J-156: A 3D Camera Test-Bed for Real-Time Monitoring and Quantitative Tracking of Radiotherapy Treatment

Purpose: To investigate methods for 3D camera‐based monitoring and quantitative tracking in the radiation therapy treatment room.Methods and Materials: A laboratory test‐bed for installing 3D cameras and testing 3D tracking algorithms was built to develop techniques for 3D treatment room monitoring. Kinect cameras were used as the 3D camera platforms. The cameras were anchored to a rigid framework attached to the laboratory walls to avoid vibrations and camera motion drift. Spatio‐temporal clustering algorithm was employed to remove camera fluttering noise. Features whose depth estimation varied by more than 3 mm in 1/3rd of a second were removed. A client server framework enabled the 3D treatment space to be visualized by remotely located experts in real‐time. A scalable multi GPU system that enabled 3D treatment space rendering in stereo and in real‐time was employed on the server side. For tracking both the patient body surface as well as the treatment room equipment, we investigated two different algorithms, using a Point Feature Histogram and multi‐resolution 3D Hough transform. For real‐time tracking we, also employed 2D multi‐resolution optical flow for effective temporal tracking on radiation equipment as well as the patient anatomy. Results: For remotely located experts, the treatment space visualization was conducted at 40 fps with a resolution of 640 × 480 pixels. Sub‐millimeter accuracy for tracking the patient body surface as well as the treatment room equipment was obtained when the tracked features were <120 cm from any of the 3D cameras, degrading with increasing distance. Conclusion: Remote real‐time stereoscopic patient setup visualization is feasible, enabling expansion of high quality radiation therapy into challenging environments. For both tested tracking approaches, the run‐time analysis showed the need for GPU hardware to track features in real‐time.

Accelerating MapReduce framework on multi-GPU systems

Graphics processors evolve rapidly and promise to support power-efficient, cost, differentiated price-performance, and scalable high performance computing. MapReduce is a well-known distributed programming model to ease the development of applications for large-scale data processing on a large number of commodity CPUs. When compared to CPUs, GPUs are an order of magnitude faster in terms of computation power and memory bandwidth, but they are harder to program. Although several studies have implemented the MapReduce model on GPUs, most of them are based on the single GPU model and bounded by a GPU memory with inefficient atomic operations. This paper focuses on the development of MGMR, a standalone MapReduce system that utilizes multiple GPUs to manage large-scale data processing beyond the GPU memory limitation, and also to eliminate serial atomic operations. Experimental results have demonstrated the effectiveness of MGMR in handling large data sets.

Skeletal based programming for dynamic programming on MultiGPU systems

Current parallel systems composed of mixed multi/manycore systems and/with GPUs become more complex due to their heterogeneous nature. The programmability barrier inherent to parallel systems increases almost with each new architecture delivery. The development of libraries, languages, and tools that allow an easy and efficient use in this new scenario is mandatory. Among the proposals found to broach this problem, skeletal programming appeared as a natural alternative to easy the programmability of parallel systems in general, but also the GPU programming in particular. In this paper, we develop a programming skeleton for Dynamic Programming on MultiGPU systems. The skeleton, implemented in CUDA, allows the user to execute parallel codes for MultiGPU just by providing sequential C++ specifications of her problems. The performance and easy of use of this skeleton has been tested on several optimization problems. The experimental results obtained over a cluster of Nvidia Fermi prove the advantages of the approach.

Multi-GPU based on multicriteria optimization for motion estimation system

Graphics processor units (GPUs) offer high performance and power efficiency for a large number of data-parallel applications. Previous research has shown that a GPU-based version of a neuromorphic motion estimation algorithm can achieve a ×32 speedup using these devices. However, the memory consumption creates a bottleneck due to the expansive tree of signal processing operations performed. In the present contribution, an improvement in memory reduction was carried out, which limited accelerator viability usage. An evolutionary algorithm was used to find the best configuration. It supposes a trade-off solution between consumption resources, parallel efficiency, and accuracy. A multilevel parallel scheme was exploited: grain level by means of multi-GPU systems, and a finer level by data parallelism. In order to achieve a more relevant analysis, some optical flow benchmarks were used to validate this study. Satisfactory results opened the chance of building an intelligent motion estimation system that auto-adapted according to real-time, resource consumption, and accuracy requirements.

GPU-Accelerated Visualization of Scattered Point Data

As data sets continue to grow in size, visualization has become a vitally important tool for extracting meaningful knowledge. Scattered point data, which are unordered sets of point coordinates with associated measured values, arise in many contexts, such as scientific experiments, sensor networks, and numerical simulations. In this paper, we present a method for visualizing such scattered point data sets. Our method is based on volume ray casting, and distinguishes itself by operating directly on the unstructured samples, rather than resampling them to form voxels. We estimate the intensity of the volume at points along the rays by interpolation using nearby samples, taking advantage of an octree to facilitate efficient range search. The method has been implemented on multi-core CPUs, GPUs as well as multi-GPU systems.1 To test our method, actual X-ray diffraction data sets have been used, consisting of up to 240 million data points. We are able to generate images of good quality and achieve interactive frame rates in favorable cases. The GPU implementation (Nvidia Tesla K20) achieves speedups of 8-14 compared with our parallelized CPU version (4-core, hyperthreaded Intel i7 3770K).

GPU-Chariot: A Programming Framework for Stream Applications Running on Multi-GPU Systems

SUMMARY This paper presents a stream programming framework, named GPU-chariot, for accelerating stream applications running on graphics processing units (GPUs). The main contribution of our framework is that it realizes efficient software pipelines on multi-GPU systems by enabling out-of-order execution of CPU functions, kernels, and data transfers. To achieve this out-of-order execution, we apply a runtime scheduler that not only maximizes the utilization of system resources but also encapsulates the number of GPUs available in the system. In addition, we implement a load-balancing capability to flow data efficiently through multiple GPUs. Furthermore, a callback interface enables overlapping execution of functions in third-party libraries. By using kernels with different performance bottlenecks, we show that our out-of-order execution is up to 20% faster than in-order execution. Finally, we conduct several case studies on a 4-GPU system and demonstrate the advantages of GPU-chariot over a manually pipelined code. We conclude that GPU-chariot can be useful when developing stream applications with software pipelines on multiple

An optimal k-exclusion real-time locking protocol motivated by multi-GPU systems

An optimal k-exclusion real-time locking protocol motivated by multi-GPU systems

Parallel Implementations of FGMRES for Solving Large, Sparse Non-symmetric Linear Systems

The Flexible Generalized Minimal Residual method (FGMRES) is an attractive iterative solver for non-symmetric systems of linear equations. This paper presents several methods for parallelizing FGMRES for a variety of archi- tectures including multi-core CPU, Graphics Processing Units (GPU), and multi-GPU systems. The parallel imple- mentations utilize OpenMP and CUDA kernels, and are organized according to thread scope. The linear systems employed in this study correspond to the discrete analogues of realistic three-dimensional convection-diffusion problems, and range in size to nearly 107 linear equations. All of the parallel implementations, particularly the novel hybrid approach, show a significant speedup over the sequential version. Theoretical insight and perfor- mance data is provided to allow informed decisions as to the most effective parallelization method for a given architecture.

High-level Programming for Medical Imaging on Multi-GPU Systems Using the SkelCL Library

Application development for modern high-performance systems with Graphics Processing Units (GPUs) relies on low-level programming approaches like CUDA and OpenCL, which leads to complex, lengthy and error-prone programs.In this paper, we present SkelCL – a high-level programming model for systems with multiple GPUs and its implementa- tion as a library on top of OpenCL. SkelCL provides three main enhancements to the OpenCL standard: 1) computations are conveniently expressed using parallel patterns (skeletons); 2) memory management is simplified using parallel container data types; 3) an automatic data (re)distribution mechanism allows for scalability when using multi-GPU systems.We use a real-world example from the field of medical imaging to motivate the design of our programming model and we show how application development using SkelCL is simplified without sacrificing performance: we were able to reduce the code size in our imaging example application by 50% while introducing only a moderate runtime overhead of less than 5%.

An optimal k-exclusion real-time locking protocol motivated by multi-GPU systems

Graphics processing units (GPUs) are becoming increasingly important in today's platforms as their growing generality allows for them to be used as powerful co-processors. In previous work, the authors showed that GPUs may be integrated into real-time systems by treating GPUs as shared resources, allocated to real-time tasks through mutual exclusion locking protocols. In this paper, an asymptotically optimal k-exclusion locking protocol is presented for globally-scheduled job-level static-priority (JLSP) systems. This protocol may be used to manage a pool of resources, such as GPUs, in such systems.

Global Shared Memory Design for Multi-GPU Graphics Cards on Personal Supercomputer

When programming CUDA or OpenCL on multi-GPU systems, the programmers usually expect the GPUs on the same system can communicate fast with each other. For instance, they hope a device memory copy from GPU1s memory to GPU2s memory can be done inside the graphics card, and needn’t to employ the PCIE, which is in relative low speed. In this paper, we propose an idea to add a multi-channel memory to the multi-GPU board, and this memory is only for transferring data between different GPUs. This multi-channel memory should have multiple interfaces, including one common interface shared by different GPUs, which is connected with a FPGA arbitration circuit and several other interfaces connected with dedicated GPUs frame buffer independently. To distinguish the shared memory of a stream multiprocessor, we call this memory Global Shared Memory. We analyze the performance improvement expectation with this global shared memory, with the case of accelerating computer tomography algebraic reconstruction on multi-GPU.

Evaluation of likelihood functions on CPU and GPU devices

We describe parallel implementations of an algorithm used to evaluate the likelihood function used in data analysis. The implementations run, respectively, on CPU and GPU, and both devices cooperatively (hybrid). CPU and GPU implementations are based on OpenMP and OpenCL, respectively. The hybrid implementation allows the application to run also on multi-GPU systems (not necessarily of the same type). The hybrid case uses a scheduler so that the workload needed for the evaluation of function is split and balanced in corresponding sub-workloads to be executed in parallel on each device, i. e. CPU-GPU or multi-CPUs. We present the results of the scalability when running on CPU. Then we show the comparison of the performance of the GPU implementation on different hardware systems from different vendors, and the performance when running in the hybrid case. The tests are based on likelihood functions from real data analysis carried out in the high energy physics community.

Scalable framework for mapping streaming applications onto multi-GPU systems

Graphics processing units leverage on a large array of parallel processing cores to boost the performance of a specific streaming computation pattern frequently found in graphics applications. Unfortunately, while many other general purpose applications do exhibit the required streaming behavior, they also possess unfavorable data layout and poor computation-to-communication ratios that penalize any straight-forward execution on the GPU. In this paper we describe an efficient and scalable code generation framework that can map general purpose streaming applications onto a multi-GPU system. This framework spans the entire core and memory hierarchy exposed by the multi-GPU system. Several key features in our framework ensure the scalability required by complex streaming applications. First, we propose an efficient stream graph partitioning algorithm that partitions the complex application to achieve the best performance under a given shared memory constraint. Next, the resulting partitions are mapped to multiple GPUs using an efficient architecture-driven strategy. The mapping balances the workload while considering the communication overhead. Finally, a highly effective pipeline execution is employed for the execution of the partitions on the multi-GPU system. The framework has been implemented as a back-end of the StreamIt programming language compiler. Our comprehensive experiments show its scalability and significant performance speedup compared with a previous state-of-the-art solution.