Sparse Matrix-vector Multiplication Research Articles

Improved implementation of concentrated plasticity models in real-time hybrid simulation

Real-time hybrid simulation (RTHS) is a cost-effective experimental testing technique for evaluating the dynamic response of structural systems. In RTHS, a portion of the structure is modeled numerically (i.e., numerical substructure), while the remaining part is tested physically (i.e., experimental substructure). These substructures are coupled in real time. A significant challenge in RTHS is accurately computing the dynamic response of the numerical substructure in real time during testing. Constrained by this challenge, numerical substructures in RTHS are often oversimplified (e.g., spring-mass-dashpot models), failing to capture more complex dynamic behaviors and limiting the applicability of RTHS. However, with the increasing use of innovative structural systems, new materials, and advanced design guidelines to ensure resilient and sustainable civil infrastructure against natural hazards, the demand for real-time executable higher-fidelity numerical substructures in RTHS is growing. Consequently, concentrated plasticity model (CPM) has gained attention as the numerical substructure in RTHS for its good balance between numerical accuracy and computational efficiency, as well as its popularity in structural engineering applications. Currently, static condensation is utilized to prevent an increase in computational degrees of freedom (DOFs) induced by the hinges in CPM, aiming to save computational cost. However, this method proves counterproductive for nonlinear analyses. While reducing the number of computational DOFs, it complicates the stiffness representation, thereby necessitating more demanding arithmetic operations for stiffness updating. In this study, multi-point constraints are applied to enhance the computational efficiency, specifically for nonlinear analyses involving stiffness updating, and the versatility of CPM. Moreover, direct stiffness updating method and sparse matrix-vector multiplication are introduced for further enhancement in computation. An open-source MATLAB/Simulink-based computational tool, namely the Hybrid2D, is developed to promote the use of the improved CPMs in RTHS as numerical substructures. Numerical simulations are first conducted to verify the accuracy and the computational efficiency of the improved CPMs in Hybrid2D. Subsequently, two RTHS examples are experimentally conducted to demonstrate the effectiveness of the Hybrid2D in enabling RTHS with a higher-fidelity numerical substructure for more complex structural systems.

Toward Energy-efficient STT-MRAM-based Near Memory Computing Architecture for Embedded Systems

Convolutional Neural Networks (CNNs) have significantly impacted embedded system applications across various domains. However, this exacerbates the real-time processing and hardware resource-constrained challenges of embedded systems. To tackle these issues, we propose spin-transfer torque magnetic random-access memory (STT-MRAM)-based near memory computing (NMC) design for embedded systems. We optimize this design from three aspects: Fast-pipelined STT-MRAM readout scheme provides higher memory bandwidth for NMC design, enhancing real-time processing capability with a non-trivial area overhead. Direct index compression format in conjunction with digital sparse matrix-vector multiplication (SpMV) accelerator supports various matrices of practical applications that alleviate computing resource requirements. Custom NMC instructions and stream converter for NMC systems dynamically adjust available hardware resources for better utilization. Experimental results demonstrate that the memory bandwidth of STT-MRAM achieves 26.7 GB/s. Energy consumption and latency improvement of digital SpMV accelerator are up to 64× and 1,120× across sparsity matrices spanning from 10% to 99.8%. Single-precision and double-precision elements transmission increased up to 8× and 9.6×, respectively. Furthermore, our design achieves a throughput of up to 15.9× over state-of-the-art designs.

Leveraging index compression techniques to optimize the use of co-processors


 
 
 The significant presence that many-core devices like GPUs have these days, and their enormous computational power, motivates the study of sparse matrix operations in this hardware. The essential sparse kernels in scientific computing, such as the sparse matrix-vector multiplication (SpMV), usually have many different high-performance GPU implementations. Sparse matrix problems typically imply memory-bound operations, and this characteristic is particularly limiting in massively parallel processors. This work revisits the main ideas about reducing the volume of data required by sparse storage formats and advances in understanding some compression techniques. In particular, we study the use of index compression combined with sparse matrix reordering techniques in CSR and explore other approaches using a blocked format. The systematic experimental evaluation on a large set of real-world matrices confirms that this approach achieves meaningful data storage reductions. Additionally, we find promising results of the impact of the storage reduction on the execution time when using accelerators to perform the mathematical kernels.
 
 

Block-wise dynamic mixed-precision for sparse matrix-vector multiplication on GPUs

AbstractSparse matrix-vector multiplication (SpMV) plays a critical role in a wide range of linear algebra computations, particularly in scientific and engineering disciplines. However, the irregular memory access patterns, extensive memory usage, high bandwidth requirements, and underutilization of parallelism hinder the computational efficiency of SpMV on GPUs. In this paper, we propose a novel approach called block-wise dynamic mixed-precision (BDMP) to address these challenges. Our methodology involves partitioning the original matrix into uniformly sized blocks, with each block’s size determined by considering architectural characteristics and accuracy requirements. Additionally, we dynamically assign precision to each block using a precision selection method that takes into account the value distribution of the original sparse matrix. We develop two distinct SpMV computation algorithms for BDMP: BDMP-PBP (Precision-based partitioning) and BDMP-TCKI (Tailored compression and kernel implementation). BDMP-PBP partitions the matrix into two independent matrices for separate computations based on block precision, offering flexibility for integration with other optimization techniques. Meanwhile, BDMP-TCKI focuses on achieving significant thread-level parallelism and memory utilization by tailoring an appropriate compressed storage format and kernel implementation for each block. We compare BDMP with NVIDIA’s cuSPARSE library and three state-of-the-art SpMV methods, including SELLP, MergeBase, and BalanceCSR, using matrices from the University of Florida’s SuiteSparse dataset collection. BDMP-PBP and BDMP-TCKI show average speedups up to 2.64$$\times $$ × and 2.91$$\times $$ × on Turing RTX 2080Ti, and up to 2.99$$\times $$ × and 3.22$$\times $$ × on Ampere A100. The results demonstrate that BDMP enables the optimization of computation speed without compromising the precision necessary for reliable results.

Iterative eigensolver using fixed-point photonic primitive.

Photonic computing has potential advantages in speed and energy consumption yet is subject to inaccuracy due to the limited equivalent bitwidth of the analog signal. In this Letter, we demonstrate a configurable, fixed-point coherent photonic iterative solver for numerical eigenvalue problems using shifted inverse iteration. The photonic primitive can accommodate arbitrarily sized sparse matrix-vector multiplication and is deployed to solve eigenmodes in a photonic waveguide structure. The photonic iterative eigensolver does not accumulate errors from each iteration, providing a path toward implementing scientific computing applications on photonic primitives.

Efficient FPGA-Based Sparse Matrix–Vector Multiplication With Data Reuse-Aware Compression

Sparse matrix-vector multiplication (SpMV) on FPGAs has gained much attention. The performance of SpMV is mainly determined by the number of multiplications between non-zero matrix elements and the corresponding vector values per cycle. On the one side, the off-chip memory bandwidth limits the number of non-zero matrix elements transferred from the off-chip DDR to the FPGA chip per cycle. On the other side, the irregular vector access pattern poses challenges to fetch the corresponding vector values. Besides, the read-after-write (RAW) dependency in the accumulation process shall be solved to enable a fully pipelined design. In this work, we propose an efficient FPGA-based sparse matrix-vector multiplication accelerator with data reuse-aware compression. The key observation is that repeated accesses to a vector value can be omitted by reusing the fetched data. Based on the observation, we propose a reordering algorithm to manually exploit the data reuse of fetched vector values. Further, we propose a novel compressed format called data reuse-aware compressed (DRC) to take full advantage of the data reuse and a fast format conversion algorithm to shorten the preprocessing time. Meanwhile, we propose an HLSfriendly accumulator to solve the RAW dependency. Finally, we implement and evaluate our proposed design on the Xilinx Zynq-UltraScale ZCU106 platform with a set of sparse matrices from the SuiteSparse matrix collection. Our proposed design achieves an average 1.18x performance speedup without the DRC format and an average 1.57x performance speedup with the DRC format w.r.t. the state-of-the-art work respectively.

Revisiting thread configuration of SpMV kernels on GPU: A machine learning based approach

Sparse matrix-vector multiplication (SpMV) optimization on GPUs has been challenging due to irregular memory accesses and unbalanced workloads. The majority of existing solutions assign a fixed number of threads to one or more rows of sparse matrices according to empirical formulas. However, this method does not give the optimal thread configuration and results in a significant performance loss. This paper proposes a new machine learning-based thread assignment strategy for SpMV on GPU, predicting the near-optimal thread configuration for matrices. Further, we partition irregular sparse matrices into blocks according to the distribution of non-zero elements and predict the optimal thread configuration for each block. A new SpMV kernel is designed to accelerate the execution of different blocks. Experimental results show that our machine learning-based approach can select the near-optimal thread configuration for most matrices. The efficiency of SpMV for irregular matrices is also improved by matrix partitioning and blockwise prediction. Finally, we dive into the trained model to find out the connection between the features of a sparse matrix and its optimal thread configuration.

Optimizing the Performance of the Sparse Matrix-Vector Multiplication Kernel in FPGA Guided by the Roofline Model.

The widespread adoption of massively parallel processors over the past decade has fundamentally transformed the landscape of high-performance computing hardware. This revolution has recently driven the advancement of FPGAs, which are emerging as an attractive alternative to power-hungry many-core devices in a world increasingly concerned with energy consumption. Consequently, numerous recent studies have focused on implementing efficient dense and sparse numerical linear algebra (NLA) kernels on FPGAs. To maximize the efficiency of these kernels, a key aspect is the exploration of analytical tools to comprehend the performance of the developments and guide the optimization process. In this regard, the roofline model (RLM) is a well-known graphical tool that facilitates the analysis of computational performance and identifies the primary bottlenecks of a specific software when executed on a particular hardware platform. Our previous efforts advanced in developing efficient implementations of the sparse matrix-vector multiplication (SpMV) for FPGAs, considering both speed and energy consumption. In this work, we propose an extension of the RLM that enables optimizing runtime and energy consumption for NLA kernels based on sparse blocked storage formats on FPGAs. To test the power of this tool, we use it to extend our previous SpMV kernels by leveraging a block-sparse storage format that enables more efficient data access.

An efficient framework for matrix-free SpMV computation on GPU for elastoplastic problems

High computational cost in elastoplastic analysis is often handled by the use of high performance parallel computers. However, the presence of both elastic and plastic states leads to the branching issue, which prevents the realization of true parallel performance. The computational efficiency of an elastoplastic analysis using finite element method is largely determined by the performance of repeated solution of linearized system of equations. In this paper, we propose GPU-based matrix-free strategies to compute sparse matrix–vector multiplication (SpMV) in Conjugate Gradient (CG) iterative solver for the acceleration of solution of linear system of equations. Matrix-free solvers never assemble large sparse global tangent matrix and perform the computation directly with small dense elemental matrices, reducing the storage requirement and preventing the use of problematic sparse storage formats. A uniform treatment of elements in elastic and plastic regions is achieved by the proposed single kernel strategy, which prevents branching, avoids redundant computation and provides efficient memory access. In addition, we propose node-based and degrees-of-freedom (DOF)-based parallel strategies for effective implementation of matrix-free SpMV on a GPU. The proposed strategies use single elemental tangent matrix for all elements in elastic region and individual tangent matrices in plastic region. The computational experiments over three large-scale benchmark examples of elastoplasticity reveal that the performance of the node-based and DOF-based parallelization strategies depends on the amount of plasticity in the body. At low plasticity levels, node-based strategy performs best, achieving 3.2× speedup over an existing GPU-based matrix-free SpMV strategy in the literature. For moderate to high amount of plasticity, the DOF-based strategy outperforms every other strategy and obtains speedups of up to 3.5× over the existing SpMV strategy.

Leveraging Memory Copy Overlap for Efficient Sparse Matrix-Vector Multiplication on GPUs

Sparse matrix-vector multiplication (SpMV) is central to many scientific, engineering, and other applications, including machine learning. Compressed Sparse Row (CSR) is a widely used sparse matrix storage format. SpMV using the CSR format on GPU computing platforms is widely studied, where the access behavior of GPU is often the performance bottleneck. The Ampere GPU architecture recently from NVIDIA provides a new asynchronous memory copy instruction, memcpy_async, for more efficient data movement in shared memory. Leveraging the capability of this new memcpy_async instruction, we first propose the CSR-Partial-Overlap to carefully overlap the data copy from global memory to shared memory and computation, allowing us to take full advantage of the data transfer time. In addition, we design the dynamic batch partition and the dynamic threads distribution to achieve effective load balancing, avoid the overhead of fixing up partial sums, and improve thread utilization. Furthermore, we propose the CSR-Full-Overlap based on the CSR-Partial-Overlap, which takes the overlap of data transfer from host to device and SpMV kernel execution into account as well. The CSR-Full-Overlap unifies the two major overlaps in SpMV and hides the computation as much as possible in the two important access behaviors of the GPU. This allows CSR-Full-Overlap to achieve the best performance gains from both overlaps. As far as we know, this paper is the first in-depth study of how memcpy_async can be potentially applied to help accelerate SpMV computation in GPU platforms. We compare CSR-Full-Overlap to the current state-of-the-art cuSPARSE, where our experimental results show an average 2.03x performance gain and up to 2.67x performance gain.

A Survey of Accelerating Parallel Sparse Linear Algebra

Sparse linear algebra includes the fundamental and important operations in various large-scale scientific computing and real-world applications. There exists performance bottleneck for sparse linear algebra since it mainly contains the memory-bound computations with low arithmetic intensity. How to improve its performance has increasingly become a focus of research efforts. Using parallel computing techniques to accelerate sparse linear algebra is currently the most popular method, while facing various challenges, e.g., large-scale data brings difficulties in storage, and the sparsity of data leads to irregular memory accesses and parallel load imbalance. Therefore, this article provides a comprehensive overview on acceleration of sparse linear algebra operations using parallel computing platforms, where we focus on four main classifications: sparse matrix-vector multiplication (SpMV), sparse matrix-sparse vector multiplication (SpMSpV), sparse general matrix-matrix multiplication (SpGEMM), and sparse tensor algebra. The takeaways from this article include the following: understanding the challenges of accelerating linear sparse algebra on various hardware platforms; understanding how structured data sparsity can improve storage efficiency; understanding how to optimize parallel load balance; understanding how to improve the efficiency of memory accesses; understanding how do the adaptive frameworks automatically select the optimal algorithms; and understanding recent design trends for acceleration of parallel sparse linear algebra.

Improving Structured Grid-Based Sparse Matrix-Vector Multiplication and Gauss–Seidel Iteration on GPDSP

Structured grid-based sparse matrix-vector multiplication and Gauss–Seidel iterations are very important kernel functions in scientific and engineering computations, both of which are memory intensive and bandwidth-limited. GPDSP is a general purpose digital signal processor, which is a very significant embedded processor that has been introduced into high-performance computing. In this paper, we designed various optimization methods, which included a blocking method to improve data locality and increase memory access efficiency, a multicolor reordering method to develop Gauss–Seidel fine-grained parallelism, a data partitioning method designed for GPDSP memory structures, and a double buffering method to overlap computation and access memory on structured grid-based SpMV and Gauss–Seidel iterations for GPDSP. At last, we combined the above optimization methods to design a multicore vectorization algorithm. We tested the matrices generated with structured grids of different sizes on the GPDSP platform and obtained speedups of up to 41× and 47× compared to the unoptimized SpMV and Gauss–Seidel iterations, with maximum bandwidth efficiencies of 72% and 81%, respectively. The experiment results show that our algorithms could fully utilize the external memory bandwidth. We also implemented the commonly used mixed precision algorithm on the GPDSP and obtained speedups of 1.60× and 1.45× for the SpMV and Gauss–Seidel iterations, respectively.

An adaptive approach for compression format based on bagging algorithm

ABSTRACT The traditional parallel Sparse matrix vector multiplication (SpMV) method has been optimized by an application-specific or compression format-specific. However, a single compression format cannot deal with all sparse matrix types effectively in practical applications. To solve this problem, an adaptive compression format based on Bagging ensemble learning algorithm is proposed in this paper. Experiments show that the adaptive compression format has higher prediction and computational performance on NVIDIA V100 and NVIDIA RTX 2080Ti. Compared with SpMV of the four compression formats, SpMV based on adaptive compression format reduces the execution time of 1.5×, 6.6×, 9× and 1.1×, respectively.

Fully-coupled parallel solver for the simulation of two-phase incompressible flows

In the framework of the in-house code Fugu, a fully-coupled solver is developed for massively parallel simulations of three-dimensional incompressible multiphase flows. The linearized momentum and continuity equations arising from the implicit solution of the fluid velocities and pressure are solved simultaneously. The method uses a BiCGStab(2) (Dongarra et al., 1998) iterative solver with an original preconditioner for the velocity block and an approximation of the inverse of the Schur complement. This is achieved by using PFMG or SMG from HYPRE and an efficient sparse matrix–vector multiplication using the CSR storage format. The construction and the tracking of the interface separating the different involved phases is based on a conservative VOF method. Test cases, such as a spherical bubble rising in quiescent liquid and the free fall of a dense sphere, are performed to validate the models, especially in the presence of strong density and viscosity ratios between fluids. Other cases, such as the phase inversion, demonstrate the ability of the new fully-coupled solver to solve two-phase problems with more than 1 billion degrees of freedom with excellent scalability.

Real-space solution to the electronic structure problem for nearly a million electrons.

We report a Kohn-Sham density functional theory calculation of a system with more than 200 000 atoms and 800 000 electrons using a real-space high-order finite-difference method to investigate the electronic structure of large spherical silicon nanoclusters. Our system of choice was a 20nm large spherical nanocluster with 202 617 silicon atoms and 13 836 hydrogen atoms used to passivate the dangling surface bonds. To speed up the convergence of the eigenspace, we utilized Chebyshev-filtered subspace iteration, and for sparse matrix-vector multiplications, we used blockwise Hilbert space-filling curves, implemented in the PARSEC code. For this calculation, we also replaced our orthonormalization + Rayleigh-Ritz step with a generalized eigenvalue problem step. We utilized all of the 8192 nodes (458 752 processors) on the Frontera machine at the Texas Advanced Computing Center. We achieved two Chebyshev-filtered subspace iterations, yielding a good approximation of the electronic density of states. Our work pushes the limits on the capabilities of the current electronic structure solvers to nearly 106 electrons and demonstrates the potential of the real-space approach to efficiently parallelize large calculations on modern high-performance computing platforms.

Bootstrapped block Lanczos for large-dimension eigenvalue problems

The Lanczos algorithm has proven itself to be a valuable matrix eigensolver for problems with large dimensions, up to hundreds of millions or even tens of billions. The computational cost of using any Lanczos algorithm is dominated by the number of sparse matrix-vector multiplications required to reach suitable convergence. Block Lanczos replaces sparse matrix-vector multiplication with sparse matrix-matrix multiplication; sparse matrix-matrix multiplication is more efficient, due to improved data locality and amortizing the cost of fetching or constructing a matrix element, but for a randomly chosen starting block (or pivot), block Lanczos requires more multiplications to reach convergence. We find that a bootstrapped pivot block, that is, an initial block constructed from approximate eigenvectors computed in a truncated space, leads to a dramatically reduced number of multiplications, significantly outperforming both standard vector Lanczos and block Lanczos with a random pivot. A key condition for speed-up is that the pivot block has a non-trivial overlap with the final converged vector. We implement this approach in a configuration-interaction code for nuclear structure, and find a reduction in time-to-solution by a factor of two or more, up to a factor of ten.

Solving the electronic structure problem for over 100 000 atoms in real space

Using a real-space high-order finite-difference approach, we investigate the electronic structure of large spherical silicon nanoclusters. Within Kohn-Sham density functional theory and using pseudopotentials, we report the self-consistent field convergence of a system with over 100 000 atoms: a ${\mathrm{Si}}_{107,641}{\mathrm{H}}_{9,084}$ nanocluster with a diameter of 16 nm. Our approach uses Chebyshev-filtered subspace iteration to speed up the convergence of the eigenspace, and blockwise Hilbert space-filling curves to speed up sparse matrix-vector multiplications, all of which are implemented in the parsec code. For the largest system, we utilized 2048 nodes (114 688 cores) on the Frontera machine in the Texas Advanced Computing Center. Our quantitative analysis of the electronic structure shows how it gradually approaches its bulk counterpart as a function of nanocluster size. The band gap is enlarged due to quantum confinement in nanoclusters, but decreases as the system size increases, as expected. Our work serves as a proof of concept for the capacity of the real-space approach in efficiently parallelizing very large calculations using high-performance computer platforms, which can straightforwardly be replicated in other systems with more than ${10}^{5}$ atoms.

Efficient Estimation of Pairwise Effective Resistance

Given an undirected graph G, the effective resistance r(s,t) measures the dissimilarity of node pair s,t in G, which finds numerous applications in real-world problems, such as recommender systems, combinatorial optimization, molecular chemistry, and electric power networks. Existing techniques towards pairwise effective resistance estimation either trade approximation guarantees for practical efficiency, or vice versa. In particular, the state-of-the-art solution is based on a multitude of Monte Carlo random walks, rendering it rather inefficient in practice, especially on large graphs. Motivated by this, this paper first presents an improved Monte Carlo approach, AMC, which reduces both the length and amount of random walks required without degrading the theoretical accuracy guarantee, through careful theoretical analysis and an adaptive sampling scheme. Further, we develop a greedy approach, GEER, which combines AMC with sparse matrix-vector multiplications in an optimized and non-trivial way. GEER offers significantly improved practical efficiency over AMC without compromising its asymptotic performance and accuracy guarantees. Extensive experiments on multiple benchmark datasets reveal that GEER is orders of magnitude faster than the state of the art in terms of computational time when achieving the same accuracy.

ClipSim: A GPU-friendly Parallel Framework for Single-Source SimRank with Accuracy Guarantee

SimRank is an important metric to measure the topological similarity between two nodes in a graph. In particular, single-source and top-k SimRank has numerous applications in recommendation systems, network analysis, and web mining, etc. Mathematically, given a vertex, the computation of single-machine and single-source SimRank mainly lies in matrix-matrix operations. However, it is almost impossible to directly compute on large graphs. Thus, existing works yield to two main operations: a series of random walks, and sparse matrix and dense vector multiplication operations. This brings about high computation cost for SimRank on large graphs. In real-world applications, there is always the query time and accuracy trade-off, which hinders the computation of high-precision SimRank on large-scale graphs. To handle this problem, this paper proposesClipSim, the first GPU-friendly parallel framework that accelerates the single-source SimRank on GPU with accuracy guarantee. We design a novel data structure and GPU-friendly parallel algorithms for efficient computation of all the operations of SimRank on GPU. Moreover, our theoretical derivation enables ClipSim to largely reduce the number of random walks required for each node, while maintaining the same theoretical accuracy as the state-of-the-art algorithm, ExactSim. We conduct extensive experiments on real-world and synthetic datasets to demonstrate the accuracy and efficiency of ClipSim. The results show that compared with ExactSim, ClipSim obtains single-source SimRank vectors with the same accuracy and up to 160× faster computation time.

Enabling unstructured-mesh computation on massively tiled AI processors: An example of accelerating in silico cardiac simulation

A new trend in processor architecture design is the packaging of thousands of small processor cores into a single device, where there is no device-level shared memory but each core has its own local memory. Thus, both the work and data of an application code need to be carefully distributed among the small cores, also termed as tiles. In this paper, we investigate how numerical computations that involve unstructured meshes can be efficiently parallelized and executed on a massively tiled architecture. Graphcore IPUs are chosen as the target hardware platform, to which we port an existing monodomain solver that simulates cardiac electrophysiology over realistic 3D irregular heart geometries. There are two computational kernels in this simulator, where a 3D diffusion equation is discretized over an unstructured mesh and numerically approximated by repeatedly executing sparse matrix-vector multiplications (SpMVs), whereas an individual system of ordinary differential equations (ODEs) is explicitly integrated per mesh cell. We demonstrate how a new style of programming that uses Poplar/C++ can be used to port these commonly encountered computational tasks to Graphcore IPUs. In particular, we describe a per-tile data structure that is adapted to facilitate the inter-tile data exchange needed for parallelizing the SpMVs. We also study the achievable performance of the ODE solver that heavily depends on special mathematical functions, as well as their accuracy on Graphcore IPUs. Moreover, topics related to using multiple IPUs and performance analysis are addressed. In addition to demonstrating an impressive level of performance that can be achieved by IPUs for monodomain simulation, we also provide a discussion on the generic theme of parallelizing and executing unstructured-mesh multiphysics computations on massively tiled hardware.