Sparse Storage Format Research Articles

STCO: Enhancing Training Efficiency via Structured Sparse Tensor Compilation Optimization

Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this paper, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64 ×, 5.43 ×, 4.89 ×, and 3.91 × when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for Deep Neural Network inference acceleration.

Tuning high-level synthesis SpMV kernels in Alveo FPGAs

Sparse Matrix-Vector Multiplication (SpMV) is an essential operation in scientific and engineering fields, with applications in areas like finite element analysis, image processing, and machine learning. To address the need for faster and more energy-efficient computing, this paper investigates the acceleration of SpMV through Field-Programmable Gate Arrays (FPGAs), leveraging High-Level Synthesis (HLS) for design simplicity. Our study focuses on the AMD-Xilinx Alveo U280 FPGA, assessing the performance of the SpMV kernel from Vitis Libraries, which is the state of the art on SpMV acceleration on FPGAs. We explore kernel modifications, transition to single precision, and varying partition sizes, demonstrating the impact of these changes on execution time. Furthermore, we investigate matrix preprocessing techniques, including Reverse Cuthill-McKee (RCM) reordering and a hybrid sparse storage format, to enhance efficiency. Our findings reveal that the performance of FPGA-accelerated SpMV is influenced by matrix characteristics, by smaller partition sizes, and by specific preprocessing techniques delivering notable performance improvements. By selecting the best results from these experiments, we achieved execution time enhancements of up to 3.2×. This study advances the understanding of FPGA-accelerated SpMV, providing insights into key factors that impact performance and potential avenues for further improvement.

A Hybrid Sparse-dense Defensive DNN Accelerator Architecture against Adversarial Example Attacks

Understanding how to defend against adversarial attacks is crucial for ensuring the safety and reliability of these systems in real-world applications. Various adversarial defense methods are proposed, which aim at improving the robustness of neural networks against adversarial attacks by changing the model structure, adding detection networks, and adversarial purification network. However, deploying adversarial defense methods in existing DNN accelerators or defensive accelerators leads to many key issues. To address these challenges, this article proposes sDNNGuard , an elastic heterogeneous DNN accelerator architecture that can efficiently orchestrate the simultaneous execution of original ( target ) DNN networks and the detect algorithm or network. It not only supports for dense DNN detect algorithms, but also allows for sparse DNN defense methods and other mixed dense-sparse (e.g., dense-dense and sparse-dense) workloads to fully exploit the benefits of sparsity. sDNNGuard with a CPU core also supports the non-DNN computing and allows the special layer of the neural network, and used for the conversion for sparse storage format for weights and activation values. To reduce off-chip traffic and improve resources utilization, a new hardware abstraction with elastic on-chip buffer/computing resource management is proposed to achieve dynamical resource scheduling mechanism. We propose an extended AI instruction set for neural networks synchronization, task scheduling and efficient data interaction. Experiment results show that sDNNGuard can effectively validate the legitimacy of the input samples in parallel with the target DNN model, achieving an average 1.42× speedup compared with the state-of-the-art accelerators.

Leveraging index compression techniques to optimize the use of co-processors


 
 
 The significant presence that many-core devices like GPUs have these days, and their enormous computational power, motivates the study of sparse matrix operations in this hardware. The essential sparse kernels in scientific computing, such as the sparse matrix-vector multiplication (SpMV), usually have many different high-performance GPU implementations. Sparse matrix problems typically imply memory-bound operations, and this characteristic is particularly limiting in massively parallel processors. This work revisits the main ideas about reducing the volume of data required by sparse storage formats and advances in understanding some compression techniques. In particular, we study the use of index compression combined with sparse matrix reordering techniques in CSR and explore other approaches using a blocked format. The systematic experimental evaluation on a large set of real-world matrices confirms that this approach achieves meaningful data storage reductions. Additionally, we find promising results of the impact of the storage reduction on the execution time when using accelerators to perform the mathematical kernels.
 
 

An efficient framework for matrix-free SpMV computation on GPU for elastoplastic problems

High computational cost in elastoplastic analysis is often handled by the use of high performance parallel computers. However, the presence of both elastic and plastic states leads to the branching issue, which prevents the realization of true parallel performance. The computational efficiency of an elastoplastic analysis using finite element method is largely determined by the performance of repeated solution of linearized system of equations. In this paper, we propose GPU-based matrix-free strategies to compute sparse matrix–vector multiplication (SpMV) in Conjugate Gradient (CG) iterative solver for the acceleration of solution of linear system of equations. Matrix-free solvers never assemble large sparse global tangent matrix and perform the computation directly with small dense elemental matrices, reducing the storage requirement and preventing the use of problematic sparse storage formats. A uniform treatment of elements in elastic and plastic regions is achieved by the proposed single kernel strategy, which prevents branching, avoids redundant computation and provides efficient memory access. In addition, we propose node-based and degrees-of-freedom (DOF)-based parallel strategies for effective implementation of matrix-free SpMV on a GPU. The proposed strategies use single elemental tangent matrix for all elements in elastic region and individual tangent matrices in plastic region. The computational experiments over three large-scale benchmark examples of elastoplasticity reveal that the performance of the node-based and DOF-based parallelization strategies depends on the amount of plasticity in the body. At low plasticity levels, node-based strategy performs best, achieving 3.2× speedup over an existing GPU-based matrix-free SpMV strategy in the literature. For moderate to high amount of plasticity, the DOF-based strategy outperforms every other strategy and obtains speedups of up to 3.5× over the existing SpMV strategy.

Adaptive diagonal sparse matrix-vector multiplication on GPU

For diagonal sparse matrices that have many long zero sections or scatter points or diagonal deviations from the main diagonal, a great number of zeros need be filled to maintain the diagonal structure while using DIA to store them, which leads to the performance degradation of the existing DIA kernels because the padded zeros consume extra computation and memory resources. This motivates us to present an adaptive sparse matrix-vector multiplication (SpMV) for diagonal sparse matrices on the graphics processing unit (GPU), called DIA-Adaptive, to alleviate the drawback of DIA kernels for these cases. For DIA-Adaptive, there are the following characteristics: (1) two new sparse storage formats, BRCSD (Diagonal Compressed Storage based on Row-Blocks)-I and BRCSD-II, are proposed to adapt it to various types of diagonal sparse matrices besides adopting DIA, and SpMV kernels corresponding to these storage formats are presented; and (2) a search engine is designed to choose the most appropriate storage format from DIA, BRCSD-I, and BRCSD-II for any given diagonal sparse matrix; and (3) a code generator is presented to automatically generate SpMV kernels. Using DIA-Adaptive, the ideal storage format and kernel are automatically chosen for any given diagonal sparse matrix, and thus high performance is achieved. Experimental results show that our proposed DIA-Adaptive is effective, and has high performance and good parallelism, and outperforms the state-of-the-art SpMV algorithms for all test cases.

Symbolic and Numeric Kernel Division for Graphics Processing Unit-Based Finite Element Analysis Assembly of Regular Meshes With Modified Sparse Storage Formats

Abstract This paper presents an efficient strategy to perform the assembly stage of finite element analysis (FEA) on general purpose graphics processing units (GPUs). This strategy involves dividing the assembly task using symbolic and numeric kernels, and thereby reducing the complexity of the standard single-kernel assembly approach. Two sparse storage formats based on the proposed strategy are also developed by modifying the existing sparse storage formats with the intention of removing the degrees-of-freedom-based redundancies in the global matrix. The inherent problem of race condition is resolved through the implementation of coloring and atomics. The proposed strategy is compared with the state-of-the-art GPU-based and central processing unit (CPU)-based assembly techniques. These comparisons reveal a significant number of benefits in terms of reducing storage space requirements and execution time and increasing performance (GFLOPS). Moreover, using the proposed strategy, it is found that the coloring method is more effective compared to the atomics-based method for the existing as well as the modified storage formats.

An unstructured mesh control volume method for two-dimensional space fractional diffusion equations with variable coefficients on convex domains

In this paper, we propose a novel unstructured mesh control volume method to deal with the space fractional derivative on arbitrarily shaped convex domains, which to the best of our knowledge is a new contribution to the literature. Firstly, we present the finite volume scheme for the two-dimensional space fractional diffusion equation with variable coefficients and provide the full implementation details for the case where the background interpolation mesh is based on triangular elements. Secondly, we explore the property of the stiffness matrix generated by the integral of space fractional derivative. We find that the stiffness matrix is sparse and not regular. Therefore, we choose a suitable sparse storage format for the stiffness matrix and develop a fast iterative method to solve the linear system, which is more efficient than using the Gaussian elimination method. Finally, we present several examples to verify our method, in which we make a comparison of our method with the finite element method for solving a Riesz space fractional diffusion equation on a circular domain. The numerical results demonstrate that our method can reduce CPU time significantly while retaining the same accuracy and approximation property as the finite element method. The numerical results also illustrate that our method is effective and reliable and can be applied to problems on arbitrarily shaped convex domains.

Armadillo: a template-based C++ library for linear algebra

The C++ language is often used for implementing functionality that is performance and/or resource sensitive. While the standard C++ library provides many useful algorithms (such as sorting), in its current form it does not provide direct handling of linear algebra (matrix maths). Armadillo is an open source linear algebra library for the C++ language, aiming towards a good balance between speed and ease of use. Its high-level Application Programming Interface (API) is deliberately similar to the widely Matlab and Octave languages (Eaton et al. 2015), so that mathematical operations can be expressed in a familiar and natural manner. The library is useful for algorithm development directly in C++, or relatively quick conversion of research code into production environments. Armadillo provides efficient objects for vectors, matrices and cubes (third order tensors), as well as over 200 associated functions for manipulating data stored in the objects. Integer, floating point and complex numbers are supported, as well as dense and sparse storage formats. Various matrix factorisations are provided through integration with LAPACK (Demmel 1997), or one of its high performance drop-in replacements such as Intel MKL (Intel 2016) or OpenBLAS (Xianyi, Qian, and Saar 2016). It is also possible to use Armadillo in conjunction with NVBLAS to obtain GPU-accelerated matrix multiplication (NVIDIA 2015). Armadillo is used as a base for other open source projects, such as MLPACK, a C++ library for machine learning and pattern recognition (Curtin et al. 2013), and RcppArmadillo, a bridge between the R language and C++ in order to speed up computations (Eddelbuettel and Sanderson 2014). Armadillo internally employs an expression evaluator based on template metaprogramming techniques (Abrahams and Gurtovoy 2004), to automatically combine several operations in order to increase speed and efficiency. An overview of the internal architecture is given in (Sanderson 2010).

Tree-based Space Efficient Formats for Storing the Structure of Sparse Matrices

Sparse storage formats describe a way how sparse matrices are stored in a computer memory. Extensive research has been conducted about these formats in the context of performance optimization of the sparse matrix-vector multiplication algorithms, but memory efficient formats for storing sparse matrices are still under development, since the commonly used storage formats (like COO or CSR) are not sufficient. In this paper, we propose and evaluate new storage formats for sparse matrices that minimize the space complexity of information about matrix structure. The first one is based on arithmetic coding and the second one is based on binary tree format. We compare the space complexity of common storage formats and our new formats and prove that the latter are considerably more space efficient.

Performance evaluation of sparse matrix products in UPC

Unified Parallel C (UPC) is a Partitioned Global Address Space (PGAS) language whose popularity has increased during the last years owing to its high programmability and reasonable performance through an efficient exploitation of data locality, especially on hierarchical architectures like multicore clusters. However, the performance issues that arise in this language due to the irregular structure of sparse matrix operations have not yet been studied. Among them, the selection of an adequate storage format for the sparse matrices can significantly improve the efficiency of the parallel codes. This paper presents an evaluation, using UPC, of the most common sparse storage formats with different implementations of the matrix-vector and matrix-matrix products, which are key kernels in many scientific applications.

Implementation Procedures of Parallel Preconditioning with Sparse Matrix Based on FEM

A technique to assemble global stiffness matrix stored in sparse storage format and two parallel solvers for sparse linear systems based on FEM are presented. The assembly method uses a data structure named associated node at intermediate stages to finally arrive at the Compressed Sparse Row (CSR) format. The associated nodes record the information about the connection of nodes in the mesh. The technique can reduce large memory because it only stores the nonzero elements of the global stiffness matrix. This method is simple and effective. The solvers are Restarted GMRES iterative solvers with Jacobi and sparse appropriate inverse (SPAI) preconditioning, respectively. Some numerical experiments show that the both preconditioners can improve the convergence of the iterative method, and SPAI is more powerful than Jacobi in the sence of reducing the number of iterations and parallel efficiency. Both of the two solvers can be used to solve large sparse linear system.

An efficient version of the RMA-11 model

RMA-11 is a numerical model widely used for studing the transport of constituents andwater quality in rivers and estuaries. When applied to large water systems like the R´ıode la Plata, RMA-11 demands long execution times to compute a simulation. This pa-per presents the analysis of the computational efficiency for the RMA-11 applied to atransport model of the R´ıo de la Plata, and introduces a proposal for improving the effi-ciency by using high performance computing techniques. The improved implementationmodifies the linear system resolution methodology implemented in the model. A highperformance computing strategy was applied to the FRONTALL routine of the RMA-11,by changing their logical structure and using a sparse storage format. The experimentalresults obtained when solving representative test cases show a significant improvementon the performance, achieving significant gains in computational speed: the executiontime of the implemented version decreased up to one third of the time of the originalimplementation.

Effect of the storage format of sparse linear systems on parallel CFD computations

Implicit solutions of Computational Fluid Dynamics (CFD) problems require the iterative solution of very large systems of equations. This paper investigates the comparative costs of two storage formats for large sparse matrices, namely the Compressed Sparse Row (CSR) and BSR storage formats, for various solution strategies. The block structure of the BSR storage format is highly advantageous in applications which have several degrees of freedom per nodal point, such as CFD calculations. Depending on the solution strategy, overall reductions in both CPU time and memory can be as high as 30–50%. Such gains are especially appreciable in large 3D flow calculations.

ILUS: An incomplete LU preconditioner in sparse skyline format

Incomplete LU factorizations are among the most effective preconditioners for solving general large, sparse linear systems arising from practical engineering problems. This paper shows how an ILU factorization may be easily computed in sparse skyline storage format, as opposed to traditional row-by-row schemes. This organization of the factorization has many advantages, including its amenability when the original matrix is in skyline format, the ability to dynamically monitor the stability of the factorization and the fact that factorizations may be produced with symmetric structure. Numerical results are presented for Galerkin finite element matrices arising from the standard square lid-driven cavity problem. © 1997 John Wiley & Sons, Ltd.

ILUS: An incomplete LU preconditioner in sparse skyline format

Incomplete LU factorizations are among the most effective preconditioners for solving general large, sparse linear systems arising from practical engineering problems. This paper shows how an ILU factorization may be easily computed in sparse skyline storage format, as opposed to traditional row-by-row schemes. This organization of the factorization has many advantages, including its amenability when the original matrix is in skyline format, the ability to dynamically monitor the stability of the factorization and the fact that factorizations may be produced with symmetric structure. Numerical results are presented for Galerkin finite element matrices arising from the standard square lid-driven cavity problem.

A parallel implementation of tight-binding molecular dynamics

We introduce an efficient and scalable parallel implementation of tight-binding molecular dynamics which employs reordering of the atoms. This implementation also exploits the sparse character of the Hamiltonian matrix. Reordering allows our algorithm to scale well on parallel machines since it asssures data locality. A sparse storage format and a stabilized parallel Lanczos eigensolver allow consideration of large problem sizes relevant to materials science. We present a showcase application to a 1000-atom silicon sample.

A Parallel Implementation of Tight-Binding Molecular Dynamics Based on Reordering of Atoms and the Lanczos Eigen-Solver

AbstractWe introduce an efficient and scalable parallel implementation of tight-binding molecular dynamics (TBMD) which employs reordering of the atoms in order to maximize datalocality of the distributed tight-binding (TB) Hamiltonian matrix. Reordering of the atom labels allows our new algorithm to scale well on parallel machines since most of the TB hopping integrals for a given atom are local to the processing element (PE) therefore minimizing communication. The sparse storage format and the distribution of the required eigenvectors reduces memory requirements per PE. The sparse storage format and a stabilized parallel Lanczos eigen-solver allow consideration of large problem sizes relevant to materials science. In addition, the implementation allows the calculation of the full spectrum of individual eigen-values/-vectors of the TB matrix at each time-step. This feature is a key issue when the dielectric and optical response must be computed during a TBMD simulation. We present a benchmark of our code and an analysis of the overall efficiency.

A finite element solutionof three‐dimensional mixed convection gas flows in horizontal chnnels using preconditioned iterative metrix methods

AbstractAn algorithm is presented for the finite element solution of three‐dimensional mixed convection gas flows in channels heated from below. The algorithm uses Newton's method and iterative matrix methods. Two iterative solution algorithms, conjugate gradient squared (CGS) and generalized minimal residual (GMERS), are used in conjunction with a preconditioning technique that is simple to implement. The preconditioner is a subset of the full Jacobian matrix centered around the main diagonal but retaining the most fundamental axial coupling of the residual equations. A domain‐renumbering scheme that enhances the overall algorithm performance is proposed on the basis of and analysis of the preconditioner. Comparison with the frontal elimination method demonstrates that the iterative method will be faster when the front width exceeds approximately 500. Techniques for the direct assembly f the problem into a compressed sparse row storage format are demonstrated. Elimination of fixed boundary conditions is shown to decrease the size of the matrix problem by up to 30%. Finally, fluid flow solutions obtained with the numerical technique are presented. These solutions reveal complex three‐dimensional mixed convection fluid flow phenomena at low Reynolds numbers, including the reversal of the direction of longitudinal rolls in the presence of a strong recirculation in the entrance region of the channel.

High‐performance equation solvers and their impact on finite element analysis

The role of equation solvers in modern structural analysis software is described. Direct and iterative equation solvers which exploit vectorization on modern high-performance computer systems are described and compared. The direct solvers are two Cholesky factorization methods. The first method utilizes a novel variable-band data storage format to achieve very high computation rates and the second method uses a sparse data storage format designed to reduce the number od operations. The iterative solvers are preconditioned conjugate gradient methods. Two different preconditioners are included; the first uses a diagonal matrix storage scheme to achieve high computation rates and the second requires a sparse data storage scheme and converges to the solution in fewer iterations that the first. The impact of using all of the equation solvers in a common structural analysis software system is demonstrated by solving several representative structural analysis problems.