Graphics Processing Unit Cluster Research Articles

GPARS: Graph predictive algorithm for efficient resource scheduling in heterogeneous GPU clusters

Efficient resource scheduling in heterogeneous graphics processing unit (GPU) clusters are critical for maximizing system performance and optimizing resource utilization. However, prior research in resource scheduling algorithms typically employed machine learning (ML) algorithms to estimate job durations or GPU utilization in the cluster based on training progress and task speed. Regrettably, these studies often overlooked the performance variations among different GPU types within these clusters, as well as the presence of spatiotemporal correlations among jobs. To address these limitations, this paper introduces the graph predictive algorithm for efficient resource scheduling (GPARS) designed specifically for heterogeneous clusters. GPARS leverages spatiotemporal correlations among jobs and utilizes graph attention networks (GANs) for precise job duration prediction. Building upon the prediction results, we develop a dynamic objective function to allocate suitable GPU types for newly submitted jobs. By conducting a comprehensive analysis of Alibaba’s heterogeneous GPU cluster, we delve into the impact of GPU capacity and type on job completion time (JCT) and resource utilization. Our evaluation, using real traces from Alibaba and Philly, substantiates the effectiveness of GPARS. It achieves a remarkable 10.29% reduction in waiting time and an average improvement of 7.47% in resource utilization compared to the original scheduling method. These findings underscore GPARS’s superior performance in enhancing resource scheduling within heterogeneous GPU clusters.

GPU-HADVPPM V1.0: a high-efficiency parallel GPU design of the piecewise parabolic method (PPM) for horizontal advection in an air quality model (CAMx V6.10)

Abstract. With semiconductor technology gradually approaching its physical and thermal limits, graphics processing units (GPUs) are becoming an attractive solution for many scientific applications due to their high performance. This paper presents an application of GPU accelerators in an air quality model. We demonstrate an approach that runs a piecewise parabolic method (PPM) solver of horizontal advection (HADVPPM) for the air quality model CAMx on GPU clusters. Specifically, we first convert the HADVPPM to a new Compute Unified Device Architecture C (CUDA C) code to make it computable on the GPU (GPU-HADVPPM). Then, a series of optimization measures are taken, including reducing the CPU–GPU communication frequency, increasing the data size computation on the GPU, optimizing the GPU memory access, and using thread and block indices to improve the overall computing performance of the CAMx model coupled with GPU-HADVPPM (named the CAMx-CUDA model). Finally, a heterogeneous, hybrid programming paradigm is presented and utilized with GPU-HADVPPM on the GPU clusters with a message passing interface (MPI) and CUDA. The offline experimental results show that running GPU-HADVPPM on one NVIDIA Tesla K40m and an NVIDIA Tesla V100 GPU can achieve up to a 845.4× and 1113.6× acceleration. By implementing a series of optimization schemes, the CAMx-CUDA model results in a 29.0× and 128.4× improvement in computational efficiency by using a GPU accelerator card on a K40m and V100 cluster, respectively. In terms of the single-module computational efficiency of GPU-HADVPPM, it can achieve 1.3× and 18.8× speedup on an NVIDIA Tesla K40m GPU and NVIDIA Tesla V100 GPU, respectively. The multi-GPU acceleration algorithm enables a 4.5× speedup with eight CPU cores and eight GPU accelerators on a V100 cluster.

Adopting GPU computing to support DL-based Earth science applications

ABSTRACT With the advancement of Artificial Intelligence (AI) technologies and accumulation of big Earth data, Deep Learning (DL) has become an important method to discover patterns and understand Earth science processes in the past several years. While successful in many Earth science areas, AI/DL applications are often challenging for computing devices. In recent years, Graphics Processing Unit (GPU) devices have been leveraged to speed up AI/DL applications, yet computational performance still poses a major barrier for DL-based Earth science applications. To address these computational challenges, we selected five existing sample Earth science AI applications, revised the DL-based models/algorithms, and tested the performance of multiple GPU computing platforms to support the applications. Application software packages, performance comparisons across different platforms, along with other results, are summarized. This article can help understand how various AI/ML Earth science applications can be supported by GPU computing and help researchers in the Earth science domain better adopt GPU computing (such as supermicro, GPU clusters, and cloud computing-based) for their AI/ML applications, and to optimize their science applications to better leverage the computing device.

A framework for live host-based Bitcoin wallet forensics and triage

Organised crime and cybercriminals use Bitcoin, a popular cryptocurrency, to launder money and move it across borders with impunity. The UK and other countries have legislation to recover the proceeds of crime from criminals. Recent UK case law has recognised cryptocurrency assets as property that can be seized and realised under the Proceeds of Crime Act (POCA). To seize a cryptocurrency asset generally requires access to the private key. Anecdotal evidence suggests that if cryptocurrency is not seized quickly after enforcement action has taken place, it will be transferred to other wallets making it difficult to seize at a future time. We investigate how Bitcoin could be seized from an Electrum or Ledger hardware wallet, during a law enforcement search, using live forensic techniques and a dictionary attack.We conduct a literature review examining the state-of-the-art in Bitcoin application forensics and Bitcoin wallet attacks. Concluding, that there is a gap in research on Bitcoin wallet security and that a significant proportion of the available literature comes from a small group of academics working with industry and law enforcement (Volety et al. 2019; Van Der Horst et al., 2017; Zollner et al., 2019). We then forensically examine the Electrum software wallet and the Ledger Nano S hardware wallet, to establish what artefacts can be recovered to assist in the recovery of Bitcoin from the wallets. Our main contribution is a proposed framework for Bitcoin forensic triage, a collection tool to recover Bitcoin artefacts and identifiers, and two proof of concept dictionary-attack tools written in Python and OpenCL.We then evaluate these tools to establish if an attack is practicable using a low-cost cluster of public cloud-based Graphics Processing Unit (GPU) instances. During our investigation, we find a weakness in Electrum's storage of encrypted private keys in RAM. We leverage this to make around 2.4 trillion password guesses. We also demonstrate that we can conduct 16.6 billion guesses against a password protected Ledger seed phrase.

A GPU-based general numerical framework for plasma simulations in terms of microscopic kinetic equations with full collision terms

We have proposed a general numerical framework for plasma simulations on graphics processing unit clusters based on microscopic kinetic equations with full collision terms. Our numerical algorithm consistently deals with both long-range (classical forces in the Vlasov term) and short-range (quantum processes in the collision term) interactions. Providing the relevant particle masses, charges and types (classical, fermionic or bosonic), as well as the external forces and the matrix elements (in the collisional integral), the algorithm consistently solves the coupled multi-particle kinetic equations. Currently, the framework is being tested and applied in the field of relativistic heavy-ion collisions; extensions to other plasma systems are straightforward. Our framework is a potential and competitive numerical platform for consistent plasma simulations.

Creating Maps of the Ligand Binding Landscape for Kinetics-Based Drug Discovery.

Simulations of ligand-protein interactions can be very useful for drug design and to gain biological insight. Full pathways of ligand-protein binding can be used to get information about ligand binding transition states, which form the rate-limiting step of the binding and release processes. However, these simulations are typically limited by the presence of large energy barriers that separate stable poses of interest. Here we describe a simulation protocol for exploring and analyzing landscapes of ligand-protein interactions that makes use of molecular docking, enhanced molecular simulation with the weighted ensemble algorithm, and network analysis. It can be accomplished using a modest cluster of graphics processing units and freely accessible software. This protocol focuses on the construction and analysis of a network model of ligand binding poses and provides links to resources that describe the other steps in more detail. The end result of this protocol is a map of the ligand-protein binding landscape that identifies transition states of the ligand binding pathway, as well as alternative bound poses that could be stabilized with modifications to the ligand.

Faster Self-Consistent Field (SCF) Calculations on GPU Clusters.

A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of one-electron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.

A multigrid local smoother approach for a domain decomposition solver over non‐matching grids

AbstractIn this paper we consider a multigrid approach for solving elliptic equations over non‐matching grids with domain decomposition methods. The domain is partitioned into subdomains with different mesh levels that do not match at the interface. The proposed algorithm searches for the global solution over different levels by projecting the residuals on the overlap region. This method is used in conjunction with a domain decomposition solver which only requires, in each iteration step, the solutions of several small local subproblems over finite element blocks. This algorithm is shown to converge to the solution of the corresponding Lagrange multiplier problem for non‐matching grids. The convergence properties of the algorithms are analyzed and numerical examples are presented. When the multigrid and domain decomposition approaches are combined, the method is shown to be reliable and easy to implement. Furthermore the local nature of the solver allows for a straightforward implementation on multiple parallel computers and graphics processing unit (GPU) clusters.

Let’s Take it Offline: Boosting Brute-Force Attacks on iPhone’s User Authentication through SCA

In recent years, smartphones have become an increasingly important storage facility for personal sensitive data ranging from photos and credentials up to financial and medical records like credit cards and person’s diseases. Trivially, it is critical to secure this information and only provide access to the genuine and authenticated user. Smartphone vendors have already taken exceptional care to protect user data by the means of various software and hardware security features like code signing, authenticated boot chain, dedicated co-processor and integrated cryptographic engines with hardware fused keys. Despite these obstacles, adversaries have successfully broken through various software protections in the past, leaving only the hardware as the last standing barrier between the attacker and user data. In this work, we build upon existing software vulnerabilities and break through the final barrier by performing the first publicly reported physical Side-Channel Analysis (SCA) attack on an iPhone in order to extract the hardware-fused devicespecific User Identifier (UID) key. This key – once at hand – allows the adversary to perform an offline brute-force attack on the user passcode employing an optimized and scalable implementation of the Key Derivation Function (KDF) on a Graphics Processing Unit (GPU) cluster. Once the passcode is revealed, the adversary has full access to all user data stored on the device and possibly in the cloud.As the software exploit enables acquisition and processing of hundreds of millions oftraces, this work further shows that an attacker being able to query arbitrary many chosen-data encryption/decryption requests is a realistic model, even for compact systems with advanced software protections, and emphasizes the need for assessing resilience against SCA for a very high number of traces.

Two Decades of 4D-QSAR: A Dying Art or Staging a Comeback?

A key question confronting computational chemists concerns the preferable ligand geometry that fits complementarily into the receptor pocket. Typically, the postulated ‘bioactive’ 3D ligand conformation is constructed as a ‘sophisticated guess’ (unnecessarily geometry-optimized) mirroring the pharmacophore hypothesis—sometimes based on an erroneous prerequisite. Hence, 4D-QSAR scheme and its ‘dialects’ have been practically implemented as higher level of model abstraction that allows the examination of the multiple molecular conformation, orientation and protonation representation, respectively. Nearly a quarter of a century has passed since the eminent work of Hopfinger appeared on the stage; therefore the natural question occurs whether 4D-QSAR approach is still appealing to the scientific community? With no intention to be comprehensive, a review of the current state of art in the field of receptor-independent (RI) and receptor-dependent (RD) 4D-QSAR methodology is provided with a brief examination of the ‘mainstream’ algorithms. In fact, a myriad of 4D-QSAR methods have been implemented and applied practically for a diverse range of molecules. It seems that, 4D-QSAR approach has been experiencing a promising renaissance of interests that might be fuelled by the rising power of the graphics processing unit (GPU) clusters applied to full-atom MD-based simulations of the protein-ligand complexes.

Real-time gradation-expressible amplitude-modulation-type electroholography based on binary-weighted computer-generated hologram

In amplitude-modulation-type electroholography, the binary-weighted computer-generated hologram (BW-CGH) facilitates the gradation-expressible reconstruction of three-dimensional (3D) objects. To realize real-time gradation-expressible electroholography, we propose an efficient and high-speed method for calculating bit planes consisting of BW-CGHs. The proposed method is implemented on a multiple graphics processing unit (GPU) cluster system comprising 13 GPUs. The proposed BW-CGH method realizes eight-gradation-expressible electroholography at approximately the same calculation speed as that of conventional electroholography based on binary computer-generated holograms. Consequently, we were able to successfully reconstruct a real-time electroholographic 3D video comprising approximately 180,000 points expressed in eight gradations at 30 frames per second.

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters.

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.

Direct numerical simulations of turbulent periodic-hill flows with mass-conserving lattice Boltzmann method

The multi-relaxation time lattice Boltzmann method is used to perform direct numerical simulations of laminar and turbulent pressure-driven flows within a channel with hill-shape periodic constriction for the first time. The simulations are conducted on a graphics processing unit cluster with two-dimensional domain decomposition to accelerate the computation. The hill-shape boundary is represented using the interpolated bounce back scheme. However, the scheme generates mass leakage across the boundary, which is more pronounced in the turbulent flow regime, and this may produce diverging solutions for turbulent flows. The mass leakage due to the local mass imbalance along the curved boundary is solved by modifying the distribution functions locally or globally, and both predict similar velocity distributions. Since the global correction method is more computing time-consuming, the local correction method is adopted. The present numerical implementation’s capability is first validated by performing direct numerical simulations of turbulent channel flow at Reτ = 180, and the current predicted results agree well with the benchmark solutions. Direct numerical simulations are further conducted for the turbulent flow over the periodic hill at Reh = 2800. Both the mean velocity and turbulent stress compare favorably with the benchmark solutions. The present simulation also correctly predicts the turbulence splatting effect near the windward hill. Both phenomena are in good accordance with the benchmark solutions.

High-performance flow classification using hybrid clusters in software defined mobile edge computing

Mobile Edge Computing (MEC) provides different storage and computing capabilities within the access range of mobile devices. This moderates the burden of offloading compute/storage-intensive processes of the mobile devices to the centralized cloud data centers. As a result, the network latency is reduced and the quality of service provided for the mobile end users is improved. Different applications benefit from the large-scale deployments of MEC servers. However, the considerable complexity of managing large scale deployments of the sheer number of applications for the millions of mobile devices is a challenge. Recently, Software Defined Networking (SDN) is leveraged to resolve the problem by providing unified and programmable interfaces for managing network devices. Most of the current SDN packet processing services are tightly dependent on the packet classification service. This primary service classifies any incoming packet based on matching a set of specific fields of its header against a flow table. Acceleration of this basic process considerably increases the performance of the SDN-based MEC. In this paper, the hierarchical tree algorithm, which is a packet classification method, is parallelized using popular platforms on a cluster of Graphics Processing Units (GPUs), a cluster of Central Processing Units (CPUs), and a hybrid cluster. The best scenario for the parallel implementation of this algorithm on the CPU cluster is that which combines OpenMP and MPI.In this case, the throughput of the classifier is 4.2 million packets per second (MPPS). On the GPU cluster, two different scenarios have been used. In the first scenario, the global memory is used to store the rules and the Hierarchical-trie of the classifier while in the second scenario we break the filter set in a way that the resulting Hierarchical-trie of each subset could be stored in the shared memory of GPU. According to the results, although the first GPU cluster scenario achieves a throughput of 29.19 MPPS and a speedup 58 times as great as the serial mode, the second scenario is 12 times faster due to using the shared memory. The best performance, however, belongs to the hybrid cluster mode. The hybrid cluster achieves a throughput of 30.59 which is 1.4 MPPS more than the GPU cluster.

A GPU-based residual network for medical image classification in smart medicine

The recent advance of high-performance computing techniques like graphics processing unit (GPU) enables large-scale deep learning models for medical image analytics in smart medicine. Smart medicine has made great progress by applying convolutional neural networks (CNNs) like ResNet and VGG-16 to medical image classification. However, various CNN models achieve very limited accuracy in some cases where multiple diseases are revealed in an X-ray image. This paper presents a variant ResNet model by replacing the global average pooling with the adaptive dropout for medical image classification. In order for the presented model to recognize multiple diseases (i.e., multi-label classification), we convert the multi-label classification to N binary classification by training the parameters of the presented model for N times. Finally, experiments are conducted on a GPU Cluster to evaluate the presented model on three datasets, namely Montgomery County chest X-ray set, Shenzhen X-ray set, and NIH chest X-ray set. The results show the presented model achieves a great performance improvement for medical image classification without a significant efficiency reduction compared to the traditional architecture and VGG-16.

Training a Deep Learning Network with an Insignificantly Small Dataset

In the last few years, Deep Learning is one of the top research areas in academia as well as in industry. Every industry is now looking for a deep learning-based solution to the problems in hand. As a researcher, learning “Deep Learning” through practical experiments will be a very challenging task. Particularly, training a deep learning network with huge amount of training data will make it impractical to do this on a normal desktop computer or laptop. Even a small-scale application in computer vision using deep learning techniques will require several days of training the deep network model on a very higher end Graphical Processing Unit (GPU) clusters or Tensor Processing Unit (TPU) clusters that makes impractical to do that research on a conventional laptop. In this work, we address the possibilities of training a deep learning network with an insignificantly small dataset. Here we mean “significantly small dataset’ as a dataset with only few images (<10) per class. Since we are going to design a prototype drone detection system which is a single class classification problem, we hereby try to train the deep learning network only with few drone images (2 images only). Our research question is: will it be possible to train a YOLO deep learning network model only with two images and achieve a descent detection accurate on a constrained test dataset of drones? This paper addresses that issue and our results prove that it is possible to train a deep learning network only with two images and achieve good performance under constrained application environments.

Level 2 Reformulation Linearization Technique–Based Parallel Algorithms for Solving Large Quadratic Assignment Problems on Graphics Processing Unit Clusters

This paper discusses efficient parallel algorithms for obtaining strong lower bounds and exact solutions for large instances of the quadratic assignment problem (QAP). Our parallel architecture is comprised of both multicore processors and compute unified device architecture–enabled NVIDIA graphics processing units (GPUs) on the Blue Waters Supercomputing Facility at the University of Illinois at Urbana–Champaign. We propose novel parallelization of the Lagrangian dual ascent algorithm on the GPUs, which is used for solving a QAP formulation based on the level-2 reformulation linearization technique. The linear assignment subproblems in this procedure are solved using our accelerated Hungarian algorithm [Date K, Rakesh N (2016) GPU-accelerated Hungarian algorithms for the linear assignment problem. Parallel Computing 57:52–72.]. We embed this accelerated dual-ascent algorithm in a parallel branch-and-bound scheme and conduct extensive computational experiments on single and multiple GPUs, using problem instances with up to 42 facilities from the quadratic assignment problem library (QAPLIB). The experiments suggest that our GPU-based approach is scalable, and it can be used to obtain tight lower bounds on large QAP instances. Our accelerated branch-and-bound scheme is able to comfortably solve Nugent and Taillard instances (up to 30 facilities) from the QAPLIB, using a modest number of GPUs.

GEYSER: 3D thermo-hydrodynamic reactive transport numerical simulator including porosity and permeability evolution using GPU clusters

GEYSER, an acronym for Graphic processing units (GPU) cluster computing for Enhanced hYdrothermal SystEms with Reactive transport, is a 3D simulator that includes porosity and permeability evolution for mass and heat transport processes in fractured geological media. The simulator also includes mass porosity and permeability evolution in response to dehydration reactions of hydrous minerals. GEYSER utilizes a finite difference scheme to solve the governing PDEs associated with 3D large-scale hydrothermal systems or geothermal reservoirs. This tool is a high performance code using GPU workstations or cluster technology. The physical processes implemented into the code are those associated with deep hydrogeological complexes where high fluid pressures generated by dehydration reactions can be sufficient to induce hydrofractures that significantly influence the porosity and permeability structures within geological formation. The governing equations are described and implemented and applied to a simplified 3D model of a magmatic intrusion at depth underlying a deep sedimentary cover. Close to ideal, weak scaling is demonstrated on GPU clusters with up to 128 GPUs. The numerical model can be used to investigate and understand coupled and time-dependent hydromechanical and thermodynamic processes at high resolution of the 3D computational domain. Applications include the hydrogeology of volcanic environments or exploitation of sediment-hosted geothermal resources. The code can also be suited for porosity and permeability evolution regarding pressure and temperature reaction rate to rock decarbonization for CO2 sequestration in deep sedimentary formations.

An fast simulation tool for fluid animation in VR application based on GPUs

Realistic and real-time simulation of fluid animation is widely used to the application of virtual reality(VR) such as VR game, special effect in film, augmented reality (AR) and so on. However, fast simulation of complex fluid animation problem such as free interaction surface and high impact requires a large number of both physical computations and time steps. It in turn leads to high computational cost. In order to improve the problem, we design a fast tool to accelerate and simulate fluid animation using multi-node graphics processing units clusters. In this paper, we present a fluid animation model tool for VR application based on multi-GPU cluster. The model method of position-based fluid (PBF) is implemented on our tool, and some strategies for GPUs optimizations are applied to parallel system based on the character of hardware. We first present an efficient data structure for speeding up memory access. Then, an optimized parallel framework is designed to get higher performance. We adjust the size of grid sptial index, reducing the access and thread synchronization during the neighborhood search, which greatly improve the efficiency on GPU. The key work of extending the PBF method from single GPU to GPU clusters, a spatial decomposition strategy is presented based on Orthogonal Recursive Bisection(ORB) model. Finally, an effective VR tool for real-time fluid animation modeling on the GPUs cluster is designed which can create various vivid animation. The performance and efficiency of our method are demonstrated using multiple VR scenes.

High-Performance Time-Series Quantitative Retrieval From Satellite Images on a GPU Cluster

The quality and accuracy of remote sensing instruments continue to increase, allowing geoscientists to perform various quantitative retrieval applications to observe the geophysical variables of land, atmosphere, ocean, etc. The explosive growth of time-series remote sensing (RS) data over large-scales poses great challenges on managing, processing, and interpreting RS ‘‘Big Data.’’ To explore these time-series RS data efficiently, in this paper, we design and implement a high-performance framework to address the time-consuming time-series quantitative retrieval issue on a graphics processing unit cluster, taking the aerosol optical depth (AOD) retrieval from satellite images as a study case. The presented framework exploits the multilevel parallelism for time-series quantitative RS retrieval to promote efficiency. At the coarse-grained level of parallelism, the AOD time-series retrieval is represented as multidirected acyclic graph workflows and scheduled based on a list-based heuristic algorithm, heterogeneous earliest finish time, taking the idle slot and priorities of retrieval jobs into account. At the fine-grained level, the parallel strategies for the major remote sensing image processing algorithms divided into three categories, i.e., the point or pixel-based operations, the local operations, and the global or irregular operations have been summarized. The parallel framework was implemented with message passing interface and compute unified device architecture, and experimental results with the AOD retrieval case verify the effectiveness of the presented framework.