Graphics Processing Unit Acceleration Research Articles

A single-phase GPU-accelerated surface tension model using SPH

This paper presents an accelerated single-phase surface tension smoothed particle hydrodynamics (SPH) solver developed to run entirely on a graphics processing unit (GPU) capable of simulating millions of particles in three dimensions on a single GPU. The single-phase surface tension model is augmented with a contact line force to improve the prediction of the physics at the liquid-solid contact point. The surface tension model uses the modified dynamic boundary condition (mDBC) to impose no-slip conditions at the wall boundary. To enable simulations with millions of particles, the single-phase surface tension model has been implemented in the open-source SPH code DualSPHysics to exploit the GPU acceleration, paying special attention to the size of the kernel support and integration of the neighbour lists. The new scheme is validated using 2-D and 3-D test cases including drop deformation, drop oscillation, Rayleigh-Plateau instability and surface contact angles. The results show a good agreement with the analytical solutions with a standard spatial convergence behaviour. Profiling the new surface tension solver shows an additional computational complexity. The performance analysis shows that the new code has a speed up of up two orders of magnitude (x70-80) compared to the CPU-only code. Profiling the new CUDA kernels shows they have the near identical performance metrics with the main CUDA kernels in the original DualSPHysics solver.

A 1D-2D dynamic bidirectional coupling model for high-resolution simulation of urban water environments based on GPU acceleration techniques

Urban water pollution is an important issue of global concern. An integrated model that can be employed to efficiently and accurately simulate the whole system of urban surfaces, underground drainage systems and river and lake water environments is an important tool for comprehensively treating urban water pollution. Therefore, we developed a 1D-2D dynamic bidirectional coupling model for high-resolution simulation of the entire urban water environment (GAST-GCSE) comprising the graphics processing unit (GPU)-accelerated surface water flow and transport (GAST) model and the storm water management model (SWMM). The SWMM and GAST model were used for coupled simulation of hydrological and hydrodynamic quantities and water quality in 1D and 2D regions, respectively. The time steps of the two models are the same. Compute Unified Device Architecture (CUDA) parallel architecture programming and advanced model algorithms were adopted to improve the simulation accuracy and efficiency of the GAST model. The two models were dynamically coupled by calling the dynamic link library (DLL) in the SWMM and adding external functions. Two test cases were considered to verify the performance of the proposed model. The results showed that the proposed model provides high simulation accuracy. Compared with the results obtained by commercial software and theoretical values, the relative errors of the water quantity and water quality simulation results were less than 3%. The whole water environment of the main urban area of Changzhi city in North China was evaluated by the GAST-GCSE model, and the simulation results were compared with monitoring data and pure SWMM simulation results. The simulation errors at three waterlogging points were less than 14%, and the Nash efficiency coefficients (NSE) of the NH4+-N concentration in the three river sections were 0.77, 0.56 and 0.64, respectively. Notably, the model needed only 4.39 h to simulate rainfall production and confluence on a 2D surface, a river flood in complex terrain and its accompanying pollutant evolution process involving 3,849,428 cells over 4 h. This study provides new insights, ideas and tools for urban water environment simulation and change mechanism study.

GPU acceleration of conjugate gradient method obtaining Green's function for transport-property calculation

We present a graphics processing unit (GPU) acceleration procedure for the transport-property calculation of the RSPACE code, which uses density functional theory and the real-space finite-difference method. Porting the RSPACE code to run on a GPU using Open Accelerator and libraries results in the improvement of the computational speed and efficiency compared with a central processing unit. Furthermore, we achieved a speed up of ∼ 4x compared with a single GPU and effective processing in a multi-GPU environment by parallelizing in a way that minimizes communication and synchronization. Our GPU-acceleration procedure is demonstrated by the calculation of electron-transport properties of (4,4) carbon nanotubes (CNTs) with a 5-8-5 defect for which the calculations are suitable for parallel processing on GPUs. It is found that the information loss is strongly affected by the defect geometry when electron waves propagating in CNTs are utilized as the information transfer signal.

Development of scalable GPU-based direct whole-core depletion calculation methods

GPU (Graphical Processing Unit) acceleration methods are introduced to the depletion solver and the cross section (XS) calculation routines of the direct whole core calculation code, nTRACER, for scalable and superior computing performance. Extensive GPU acceleration besides transport solvers is necessary to resolve the performance degradation in depletion calculations caused by the newly generated nuclides in depleted fuel regions and the poor scalability. A matrix exponential solver employing the Gauss-Seidel-based Chebyshev rational approximate method is implemented to reduce branch divergence and load imbalance in GPU computing in the solution of the Bateman equation. A non-zero element major storage scheme is developed to optimize memory access efficiency on GPUs by exploiting the same sparsity pattern of the depletion matrix in all the depletion regions. XS data are serialized for effective massive parallelization of XS calculation on GPUs. A block decomposition scheme for XS calculation is implemented to reduce the global memory requirement due to the limited memory capacity of commercial GPUs. It is demonstrated that nTRACER can complete a direct whole core depletion calculation for a realistic core in a few hours on a moderate-sized GPU-based heterogeneous computing platform after extensive GPU porting to all the essential routines.

Particle-resolved thermal lattice Boltzmann simulation using OpenACC on multi-GPUs

We utilize the Open Accelerator (OpenACC) approach for graphics processing unit (GPU) accelerated particle-resolved thermal lattice Boltzmann (LB) simulation. We adopt the momentum-exchange method to calculate fluid-particle interactions to preserve the simplicity of the LB method. To address load imbalance issues, we extend the indirect addressing method to collect fluid-particle link information at each timestep and store indices of fluid-particle link in a fixed index array. We simulate the sedimentation of 4,800 hot particles in cold fluids with a domain size of 40002, and the simulation achieves 1750 million lattice updates per second (MLUPS) on a single GPU. Furthermore, we implement a hybrid OpenACC and message passing interface (MPI) approach for multi-GPU accelerated simulation. This approach incorporates four optimization strategies, including building domain lists, utilizing request-answer communication, overlapping communications with computations, and executing computation tasks concurrently. By reducing data communication between GPUs, hiding communication latency through overlapping computation, and increasing the utilization of GPU resources, we achieve improved performance, reaching 10846 MLUPS using 8 GPUs. Our results demonstrate that the OpenACC-based GPU acceleration is promising for particle-resolved thermal lattice Boltzmann simulation.

Speeding up the GENGA N-body integrator on consumer-grade graphics cards

Context. Graphics processing unit (GPU) computing has become popular due to the enormous calculation potential that can be harvested from a single card. The N-body integrator Gravitational ENcounters with GPU Acceleration (GENGA) is built to harvest the computing power from such cards, but it suffers a severe performance penalty on consumer-grade Nvidia GPUs due to their artificially truncated double precision performance. Aims. We aim to speed up GENGA on consumer-grade cards by harvesting their high single-precision performance. Methods. We modified GENGA to have the option to compute the mutual long-distance forces between bodies in single precision and tested this with five experiments. First, we ran a high number of simulations with similar initial conditions of on average 6600 fully self-gravitating planetesimals in both single and double precision to establish whether the outcomes were statistically different. These simulations were run on Tesla K20 cards. We supplemented this test with simulations that (i) began with a mixture of planetesimals and planetary embryos, (ii) planetesimal-driven giant planet migration, and (iii) terrestrial planet formation with a dissipating gas disc. All of these simulations served to determine the accuracy of energy and angular momentum conservation under various scenarios with single and double precision forces. Second, we ran the same simulation beginning with 40 000 self-gravitating planetesimals using both single and double precision forces on a variety of consumer-grade and Tesla GPUs to measure the performance boost of computing the long-range forces in single precision. Results. We find that there are no statistical differences when simulations are run with the gravitational forces in single or double precision that can be attributed to the force prescription rather than stochastic effects. The accumulations in uncertainty in energy are almost identical when running with single or double precision long-range forces. However, the uncertainty in the angular momentum using single rather than double precision long-range forces is about two orders of magnitude greater, but still very low. Running the simulations in single precision on consumer-grade cards decreases running time by a factor of three and becomes within a factor of three of a Tesla A100 GPU. Additional tuning speeds up the simulation by a factor of two across all types of cards. Conclusions. The option to compute the long-range forces in single precision in GENGA when using consumer-grade GPUs dramatically improves performance at a little penalty to accuracy. There is an additional environmental benefit because it reduces energy usage.

Wind Tunnel and Grid Resolution Effects in Large-Eddy Simulations of the High-Lift Common Research Model

This paper describes the application of a compressible large-eddy simulation (LES) solver to the flow over the High-Lift Common Research Model (CRM-HL) in landing configuration, a complex external aerodynamic flow configuration with deployed slats, flaps, a flow-through nacelle, and the associated brackets/fairings on the high-lift devices. The bulk Mach number is (0.2) and the mean-aerodynamic-chord-based Reynolds number is typical of a wind tunnel experiment (5.49E6). A key development in these simulations relative to previous work is that this work serves to establish robustness of LES methods to Reynolds number and aircraft configuration. Previous studies focused on the Japan Aerospace Exploration Agency Standard Model, which featured a nearly 3× lower Reynolds number than the present CRM-HL investigation and less aggressive wing/slat leading-edge curvature than the CRM-HL geometry, which led to lower leading-edge pressure suction peak magnitudes, both of which led to less stringent grid resolution requirements. In these LES simulations, an algebraic equilibrium wall modeling approach is employed along with a dynamic implementation of the Smagorinsky subgrid-scale model. The calculations are carried out in both a free air setting and one that includes the wind tunnel facility at seven angles of attack at five grid resolution levels, ranging from ≈10 to 1500 million control volumes. In free air, the solutions show decreasing sensitivity to the grid with each successive refinement level and systematically approach the experimental lift coefficient data as the grid is refined, with the 1.5 billion control volume case showing excellent agreement with the corrected experimental data. The simulations in both free air and in the wind tunnel predict a stall mechanism featuring a large inboard juncture stall and a nose-down break in the pitching moment curve, both of which agree with the experimental observations. The accuracy of the simulations is assessed via comparisons of integrated forces/moments, surface pressures, and surface skin friction visualizations. Graphics processing unit (GPU) acceleration of the charLES solver results in tractable turnaround times that make LES a useful tool in the aerospace industry design cycle. Recent GPU acceleration of the flow solver has made LES solutions that are highly accurate in lift/drag/moment for relevant high-lift aircraft flows achievable within about 5 h of wall time on 600 GPU cores.

InterOpt: Improved gene expression quantification in qPCR experiments using weighted aggregation of reference genes

qPCR is still the gold standard for gene expression quantification. However, its accuracy is highly dependent on the normalization procedure. The conventional method involves using the geometric mean of multiple study-specific reference genes (RGs) expression for cross-sample normalization. While research on selecting stably expressed RGs is extensive, scant literature exists regarding the optimal approach for aggregating multiple RGs into a unified RG. In this paper, we introduce a family of scale-invariant functions as an alternative to the geometric mean aggregation. Our candidate method (weighted geometric mean minimizing standard deviation) demonstrated significantly better results compared to other proposed methods. We provide theoretical and experimental support for this finding using real data from solid tumors and liquid biopsies. Moreover, the closed form and regression-based solution enable efficient computation and straightforward adoption on various platforms. All the proposed methods have been implemented within an easy-to-use R package with graphics processing unit (GPU) acceleration.

Multi-compartment head modeling in EEG: Unstructured boundary-fitted tetra meshing with subcortical structures.

This paper introduces an automated approach for generating a finite element (FE) discretization of a multi-compartment human head model for electroencephalographic (EEG) source localization. We aim to provide an adaptable FE mesh generation tool for EEG studies. Our technique relies on recursive solid angle labeling of a surface segmentation coupled with smoothing, refinement, inflation, and optimization procedures to enhance the mesh quality. In this study, we performed numerical meshing experiments with the three-layer Ary sphere and a magnetic resonance imaging (MRI)-based multi-compartment head segmentation which incorporates a comprehensive set of subcortical brain structures. These experiments are motivated, on one hand, by the sensitivity of non-invasive subcortical source localization to modeling errors and, on the other hand, by the present lack of open EEG software pipelines to discretize all these structures. Our approach was found to successfully produce an unstructured and boundary-fitted tetrahedral mesh with a sub-one-millimeter fitting error, providing the desired accuracy for the three-dimensional anatomical details, EEG lead field matrix, and source localization. The mesh generator applied in this study has been implemented in the open MATLAB-based Zeffiro Interface toolbox for forward and inverse processing in EEG and it allows for graphics processing unit acceleration.

GPU-HADVPPM V1.0: a high-efficiency parallel GPU design of the piecewise parabolic method (PPM) for horizontal advection in an air quality model (CAMx V6.10)

Abstract. With semiconductor technology gradually approaching its physical and thermal limits, graphics processing units (GPUs) are becoming an attractive solution for many scientific applications due to their high performance. This paper presents an application of GPU accelerators in an air quality model. We demonstrate an approach that runs a piecewise parabolic method (PPM) solver of horizontal advection (HADVPPM) for the air quality model CAMx on GPU clusters. Specifically, we first convert the HADVPPM to a new Compute Unified Device Architecture C (CUDA C) code to make it computable on the GPU (GPU-HADVPPM). Then, a series of optimization measures are taken, including reducing the CPU–GPU communication frequency, increasing the data size computation on the GPU, optimizing the GPU memory access, and using thread and block indices to improve the overall computing performance of the CAMx model coupled with GPU-HADVPPM (named the CAMx-CUDA model). Finally, a heterogeneous, hybrid programming paradigm is presented and utilized with GPU-HADVPPM on the GPU clusters with a message passing interface (MPI) and CUDA. The offline experimental results show that running GPU-HADVPPM on one NVIDIA Tesla K40m and an NVIDIA Tesla V100 GPU can achieve up to a 845.4× and 1113.6× acceleration. By implementing a series of optimization schemes, the CAMx-CUDA model results in a 29.0× and 128.4× improvement in computational efficiency by using a GPU accelerator card on a K40m and V100 cluster, respectively. In terms of the single-module computational efficiency of GPU-HADVPPM, it can achieve 1.3× and 18.8× speedup on an NVIDIA Tesla K40m GPU and NVIDIA Tesla V100 GPU, respectively. The multi-GPU acceleration algorithm enables a 4.5× speedup with eight CPU cores and eight GPU accelerators on a V100 cluster.

A Multi-GPU Aggregation-Based AMG Preconditioner for Iterative Linear Solvers

We present and release in open source format a sparse linear solver which efficiently exploits heterogeneous parallel computers. The solver can be easily integrated into scientific applications that need to solve large and sparse linear systems on modern parallel computers made of hybrid nodes hosting Nvidia Graphics Processing Unit (GPU) accelerators. The work extends previous efforts of some of the authors in the exploitation of a single GPU accelerator and proposes an implementation, based on the hybrid MPI-CUDA software environment, of a Krylov-type linear solver relying on an efficient Algebraic MultiGrid (AMG) preconditioner already available in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BootCMatchG</i> library. Our design for the hybrid implementation has been driven by the best practices for minimizing data communication overhead when multiple GPUs are employed, yet preserving the efficiency of the GPU kernels. Strong and weak scalability results of the new version of the library on well-known benchmark test cases are discussed. Comparisons with the Nvidia AmgX solution show a speedup, in the solve phase, up to 2.0x.

CUDA-Optimized GPU Acceleration of 3GPP 3D Channel Model Simulations for 5G Network Planning

The simulation of massive multiple-input multiple-output (MIMO) channel models is becoming increasingly important for testing and validation of fifth-generation new radio (5G NR) wireless networks and beyond. However, simulation performance tends to be limited when modeling a large number of antenna elements combined with a complex and realistic representation of propagation conditions. In this paper, we propose an efficient implementation of a 3rd Generation Partnership Project (3GPP) three-dimensional (3D) channel model, specifically designed for graphics processing unit (GPU) platforms, with the goal of minimizing the computational time required for channel simulation. The channel model is highly parameterized to encompass a wide range of configurations required for real-world optimized 5G NR network deployments. We use several compute unified device architecture (CUDA)-based optimization techniques to exploit the parallelism and memory hierarchy of the GPU. Experimental data show that the developed system achieves an overall speedup of about 240× compared to the original C++ model executed on an Intel processor. Compared to a design previously accelerated on a datacenter-class field programmable gate array (FPGA), the GPU design has a 33.3% higher single-precision performance but a 7.5% higher power consumption. The proposed GPU accelerator can provide fast and accurate channel simulations for 5G NR network planning and optimization.

Accelerating genomic workflows using NVIDIA Parabricks

BackgroundAs genome sequencing becomes better integrated into scientific research, government policy, and personalized medicine, the primary challenge for researchers is shifting from generating raw data to analyzing these vast datasets. Although much work has been done to reduce compute times using various configurations of traditional CPU computing infrastructures, Graphics Processing Units (GPUs) offer opportunities to accelerate genomic workflows by orders of magnitude. Here we benchmark one GPU-accelerated software suite called NVIDIA Parabricks on Amazon Web Services (AWS), Google Cloud Platform (GCP), and an NVIDIA DGX cluster. We benchmarked six variant calling pipelines, including two germline callers (HaplotypeCaller and DeepVariant) and four somatic callers (Mutect2, Muse, LoFreq, SomaticSniper).ResultsWe achieved up to 65 × acceleration with germline variant callers, bringing HaplotypeCaller runtimes down from 36 h to 33 min on AWS, 35 min on GCP, and 24 min on the NVIDIA DGX. Somatic callers exhibited more variation between the number of GPUs and computing platforms. On cloud platforms, GPU-accelerated germline callers resulted in cost savings compared with CPU runs, whereas some somatic callers were more expensive than CPU runs because their GPU acceleration was not sufficient to overcome the increased GPU cost.ConclusionsGermline variant callers scaled well with the number of GPUs across platforms, whereas somatic variant callers exhibited more variation in the number of GPUs with the fastest runtimes, suggesting that, at least with the version of Parabricks used here, these workflows are less GPU optimized and require benchmarking on the platform of choice before being deployed at production scales. Our study demonstrates that GPUs can be used to greatly accelerate genomic workflows, thus bringing closer to grasp urgent societal advances in the areas of biosurveillance and personalized medicine.

Convolution hierarchical deep-learning neural network (C-HiDeNN) with graphics processing unit (GPU) acceleration

Convolution hierarchical deep-learning neural network (C-HiDeNN) with graphics processing unit (GPU) acceleration

GPU acceleration of local and semilocal density functional calculations in the SPARC electronic structure code.

We present a Graphics Processing Unit (GPU)-accelerated version of the real-space SPARC electronic structure code for performing Kohn-Sham density functional theory calculations within the local density and generalized gradient approximations. In particular, we develop a modular math-kernel based implementation for NVIDIA architectures wherein the computationally expensive operations are carried out on the GPUs, with the remainder of the workload retained on the central processing units (CPUs). Using representative bulk and slab examples, we show that relative to CPU-only execution, GPUs enable speedups of up to 6× and 60× in node and core hours, respectively, bringing time to solution down to less than 30s for a metallic system with over 14 000 electrons and enabling significant reductions in computational resources required for a given wall time.

GPU acceleration of Nystrom type method for solving singular integral equations in H-polarized EM waves scattering by strip gratings

Abstract In this paper, we propose the use of the graphics processing units (GPUs) for acceleration of solution of the H-polarized electromagnetic waves scattering by strip gratings. The approach combines the boundary integral equation method with the Nystrom-type discretization of singular integral equations and is realized with the use of C++ and OpenCL. The elements of the matrix of the resulting system of equations can be calculated independently and, thus, in parallel. The calculation time of the proposed parallelized algorithm on GPU is compared to standard serial algorithm executed on CPU. We considered gratings of graphene strips in vacuum and inside a dielectric slab. The acceleration is up to 35–55 times for the gratings in vacuum and up to 80–140 times for the gratings of 5–20 strips inside the dielectric slab.

Efficient Simulation of Volumetric Deformable Objects in Unity3D: GPU-Accelerated Position-Based Dynamics

This paper proposes an efficient approach for simulating volumetric deformable objects using the Position-Based Dynamics (PBD) method. Volumetric bodies generated by TetGen are used to represent three-dimensional objects, which accurately capture complex shapes and volumes. However, when a large number of constraints are applied to the system to solve using serialized algorithms on central processing units (CPU), the computational cost can become a bottleneck of the simulation. To address this issue, the proposed implementation algorithm takes advantage of graphic processing unit (GPU) acceleration and parallel processing to improve the efficiency of the simulation. We propose two specific contributions: firstly, the use of the PBD method with volume constraint for tetrahedral elements to simulate volumetric deformable objects realistically; secondly, an efficient GPU-accelerated algorithm for implementing the PBD method that significantly improves computational efficiency. We also applied the node-centric and constraint-centric algorithms to solve the stretch constraint in the GPU-based algorithm. The implementation was performed using Unity3D. The compute shader feature of Unity3D was utilized to perform thousands of parallel computations in a single pass, making it possible to simulate large and complex objects in real-time. The performance of the simulation can be accelerated by using GPU-based methods with stretch and bending constraints, which provides significant speedup factors compared to using only the CPU for deformable objects such as Bunny, Armadillo, and Dragon. The constraint-centric and node-centric GPU approaches provide speedup factors of up to 8.9x and 8x, respectively, while the GPU-based methods with all types of constraints exhibit a slight decrease but still operate at real-time speeds. Overall, this approach enables the simulation of complex and irregular shapes with plausible and realistic results, while also achieving speed, robustness, and flexibility. Additionally, the proposed approach can be applied to general simulation and other game engines that support GPU-based acceleration.

Efficient spectral coupled boundary element method for fully nonlinear wave–structure interaction simulation

Accurately analyzing wave–structure interactions is crucial for the design and operational safety of ships and marine structures. This paper presents a fully nonlinear potential-flow approach for simulating wave–structure interactions using the newly proposed spectral coupled boundary element method (SCBEM). The SCBEM efficiently models an extensive water body that encompasses structures by establishing a boundary element method (BEM) computational domain solely around the object of interest while accurately simulating the far-field broad water by a spectral layer. To further improve efficiency, graphics processing unit acceleration is hired during iterative solving of the boundary value problem in the already small-sized interior BEM domain. Simulations are conducted to validate the accuracy of the method on cases with strong nonlinear phenomena, including wave run-up on a single cylinder, diffraction of a four-cylinder array, near-trapped modes for closely spaced columns, and gap resonance that occurred in side-by-side offloading. The wave run-up, diffraction wave pattern, near-trapped mode, and gap resonance frequency obtained by the proposed method are in good agreement with data from experiments and published literature. The quite good accuracy and the exceptional computational efficiency of the SCBEM demonstrate its promising potential for more application in practical marine problems.

A Multi-Layer Intrusion Detection System for SOME/IP-Based In-Vehicle Network.

The automotive Ethernet is gradually replacing the traditional controller area network (CAN) as the backbone network of the vehicle. As an essential protocol to solve service-based communication, Scalable service-Oriented MiddlewarE over IP (SOME/IP) is expected to be applied to an in-vehicle network (IVN). The increasing number of external attack interfaces and the protocol's vulnerability makes SOME/IP in-vehicle networks vulnerable to intrusion. This paper proposes a multi-layer intrusion detection system (IDS) architecture, including rule-based and artificial intelligence (AI)-based modules. The rule-based module is used to detect the SOME/IP header, SOME/IP-SD message, message interval, and communication process. The AI-based module acts on the payload. We propose a SOME/IP dataset establishment method to evaluate the performance of the proposed multi-layer IDS. Experiments are carried out on a Jetson Xavier NX, showing that the accuracy of AI-based detection reached 99.7761% and that of rule-based detection was 100%. The average detection time per packet is 0.3958 ms with graphics processing unit (GPU) acceleration and 0.6669 ms with only a central processing unit (CPU). After vehicle-level real-time analyses, the proposed IDS can be deployed for distributed or select critical advanced driving assistance system (ADAS) traffic for detection in a centralized layout.

Sparse tensor based nuclear gradients for periodic Hartree-Fock and low-scaling correlated wave function methods in the CP2K software package: A massively parallel and GPU accelerated implementation.

The development of novel double-hybrid density functionals offers new levels of accuracy and is leading to fresh insights into the fundamental properties of matter. Hartree-Fock exact exchange and correlated wave function methods, such as second-order Møller-Plesset (MP2) and direct random phase approximation (dRPA), are usually required to build such functionals. Their high computational cost is a concern, and their application to large and periodic systems is, therefore, limited. In this work, low-scaling methods for Hartree-Fock exchange (HFX), SOS-MP2, and direct RPA energy gradients are developed and implemented in the CP2K software package. The use of the resolution-of-the-identity approximation with a short range metric and atom-centered basis functions leads to sparsity, allowing for sparse tensor contractions to take place. These operations are efficiently performed with the newly developed Distributed Block-sparse Tensors (DBT) and Distributed Block-sparse Matrices (DBM) libraries, which scale to hundreds of graphics processing unit (GPU) nodes. The resulting methods, resolution-of-the-identity (RI)-HFX, SOS-MP2, and dRPA, were benchmarked on large supercomputers. They exhibit favorable sub-cubic scaling with system size, good strong scaling performance, and GPU acceleration up to a factor of 3. These developments will allow for double-hybrid level calculations of large and periodic condensed phase systems to take place on a more regular basis.